A Conjugate Gradient Algorithm with Function Value Information and N-Step Quadratic Convergence for Unconstrained Optimization.

It is generally acknowledged that the conjugate gradient (CG) method achieves global convergence--with at most a linear convergence rate--because CG formulas are generated by linear approximations of the objective functions. The quadratically convergent results are very limited. We introduce a new PRP method in which the restart strategy is also used. Moreover, the method we developed includes not only n-step quadratic convergence but also both the function value information and gradient value information. In this paper, we will show that the new PRP method (with either the Armijo line search or the Wolfe line search) is both linearly and quadratically convergent. The numerical experiments demonstrate that the new PRP algorithm is competitive with the normal CG method.

Introduction

Consider where f(x) : ℜ → ℜ is a continuously differentiable function. This optimization problem can be used to model other problems (see [1-5]). The iterative processes of the conjugate gradient (CG) methods are determined by where x is the k-th iteration; α is the step length, which is determined by a line search method; and d is the search direction defined by where g = g(x ) is the gradient ∇f(x ) of f(x) at the point x and β is a scalar. In addition, the parameter β varies between different CG formulas (see [6-17]etc.). This paper focuses on the PRP method designed by [13, 14] where g is the gradient ∇f(x ) of f(x) at the point x , and ‖.‖ denotes the Euclidean norm of vectors. Recently, there have been several successful research projects involving the PRP algorithm (see [7, 18–29] for details). The linear convergence of the standard PRP method is well known. If a restart strategy is used, the algorithm can be n-step quadratic convergent [30-32]; however, the quadratically convergent results are very limited. Zhang et al. [33] presented a three-term CG algorithm by modifying the PRP (MPRP) method and proved its global convergence. Moreover, Li and Tian [34] proved that the MPRP method with a restart strategy exhibited quadratic convergence under certain inexact line searches and suitable assumptions. In this paper, we focus primarily on the MPRP method. On the basis of the MPRP method [33], a new three-term CG formula is proposed in which the formula includes not only the gradient information but also the function information. By applying the conclusion of [34], we prove that the new CG method obtains global convergence for general functions and n-step quadratic convergence for uniformly convex functions. The numerical results show that the new method performs quite well. The main attributes of this method are stated below.

The new PRP CG method possesses not only the gradient information but also information about function values.

The sufficient descent property is satisfied without any conditions.

The global convergence, linear convergent rate, and n-step quadratic convergence are established.

The numerical results demonstrate that this method is competitive with the normal CG method for the given problems.

This paper is arranged as follows. In Section 2, we review our motivation, review the MPRP method of [33], and introduce our new PRP method. The r-step linear convergence of the new PRP method with the standard Armijo line search is proven in Section 3. In Section 4, we introduce a strategy for choosing the initial step length using the Armijo or Wolfe line search and provide the steps of the new PRP method. In Section 5, the n-step quadratic convergence of the given CG algorithm is established. In Section 6, various numerical experiments are performed to test the n-step quadratic convergence of the new PRP algorithm, and its performance is compared with that of the normal CG method. The conclusion is presented in the last section.

Motivation and Algorithm

In this section, we will provide our motivation based on the MPRP method [33] and the BFGS formulas.

The MPRP method of [33]

Zhang et al. [33] proposed a modified PRP method (a three-term CG method) called the MPRP method. The search direction of the MPRP method is defined by where y = g − g , This method can guarantee that d provides a descent direction of f at x , namely, d satisfies Because this method is regarded as the PRP method when the exact line search technique is used, we call it the MPRP method. This method possesses global convergence with the Armijo line search but fails using the Wolfe line search because the search direction does not belong to a trust region. The numerical results thus show that this method is effective.

Motivation based on the BFGS formula and the MPRP method

The BFGS method is one of the most effective methods for unconstrained optimization problems. Many satisfactory results concerning this method can be found in the literature (see [35-43] etc.), wherein Wei et al. [43] presented a new BFGS update generated by Taylor’s formula with and ρ = 2[f − f ] + (g + g ) s ; B is an approximation of the Hessian matrix of the objective function f, with B 0 = I. Clearly, the formula contains not only gradient value information but also function value information. Then, it might be argued that the resulting methods will substantially outperform the original method in theory. Practical computations demonstrate that this method is better than the normal BFGS method [43, 44]. Based on this method, to obtain the global convergence and the superlinear convergence of generally convex functions, Yuan and Wei [45] proposed a new quasi-Newton equation with where and can ensure that B inherits the positive definiteness of B for generally convex functions; the numerical results show that this method is better than the normal CG method.

Motivated by the above discussions and the CG Eq (5), the modified PRP formula is used to replace y by in Eq (5). Specifically, the new CG formula is defined by where We call this the MPRP* formula. Using (8), we can obtain and

Linear convergence of the MPRP* method

The Armijo line search finds a step length α = max{ρ ∣i = 0, 1, 2, ⋯} that satisfies where ρ ∈ (0,1) and . The Wolfe line search conditions are where and σ 1 < σ 2 < 1.

To establish the r-linear convergence of the MPRP* method, we make the following assumptions.

Assumption (i) f is a twice continuously differentiable, uniformly convex function, which means that there are positive constants M ≥ m > 0 such that where ∇2 f(x) denotes the Hessian of f at x.

Under Assumption (i), it is not difficult to obtain and where x* denotes the unique solution of the problem (1). Based on Assumption (i), we can easily obtain the following lemma and thus omit its proof.

Lemma 0.1 Let Assumption (i) hold, and let the sequence {x } be generated by the MPRP* method with the Armijo or Wolfe line search. We have where and . Moreover, when the Wolfe line search is used, we obtain

Lemma 0.2 Let Assumption (i) hold, and let the sequence {x } be generated by the MPRP* method with the Armijo line search or the Wolfe line search. Then, there is a constant c > 0 satisfying

Proof. Let where Using the definition of we obtain

Case 1: If ρ ≤ 0, we have .

Using the mean-value theorem, we have and

Moreover, we obtain Thus, it follows from (8) and the Lipschitz continuity of g that

Case 2: If ρ > 0, we have By and we obtain Then, we have and Therefore, it follows from (8) and the Lipschitz continuity of g that If the Armijo line search is used, and α ≠ 1, then such that By the mean-value theorem and the above relation, we can deduce that there exists a scalar μ ∈ (0,1) satisfying Thus, the relation holds. By the relation , we can find that Then, we obtain Setting , we have Eq (16). If the Wolfe line search is used, using Eq (13), we have By analyzing the Armijo line search technique in a similar manner, we can find a lower positive bound of the step size α . The proof is complete.

Similar to [34], we can establish the following theorem. Here, we state the theorem below but omit its proof.

Theorem 0.1 Let Assumption (i) hold, and let the sequence {x } be generated by the MPRP* method with the Armijo line search technique or Wolfe line search technique. Then, there are constants a > 0 and r ∈ (0,1) that satisfy

The restart MPRP* method and its convergence

As with [34], we define an initial step length as follows:

using the integer sequence {ϵ } → 0 with k → ∞. Moreover, we can obtain

Theorem 0.2 Let sequence {x } be generated by the MPRP* method, and let Assumption (i) hold. Then, for sufficiently large k, satisfies the Armijo line search and the Wolfe-Powell line search conditions.

Theorem 0.2 shows that, for sufficiently large k, can be defined by Eq (19) such that the Armijo line search and Wolfe-Powell line search conditions are satisfied. In the following, ∣γ ∣ is used as the initial step length of the restart MPRP* method.

Algorithm 4.1 (RMPRP*)

Step 0: Given , ρ ∈ [0,1), ϵ ∈ [0,1), and γ > 0 is an integer, let k: = 0.

Step 1: If ‖g ‖ ≤ ϵ, stop.

Step 2: If the inequality holds, we set α = ∣γ ∣; otherwise, we determine α = max{∣γ ∣ρ ∣j = 0, 1, 2, ⋯} satisfying

Step 3: Let , and k: = k+1.

Step 4: If ‖g ‖ ≤ ϵ, stop.

Step 5: If k = γ, we let x 0: = x . Go to step 1.

Step 6: Compute using (8). Go to step 2.

We now establish the global convergence of Algorithm 4.1.

Theorem 0.3 Let the conditions of Theorem 0.2 hold. Then, the following relation holds.

Proof. We will prove this theorem by contradiction. Suppose that Eq (21) does not hold; then, for all k, there exists a constant ɛ 1 > 0 satisfying Using Eqs (10) and (20), if f is bounded from below, we can obtain In particular, we have If lim α > 0, by Eqs (10) and (24), we obtain lim‖g ‖ = 0. This contradicts Eqs (22) and (21) holds. Otherwise, if lim α = 0., then there is an infinite index set K 0 satisfying According to Step 2 of Algorithm 4.1, when k is sufficiently large, α ρ −1 does not satisfy Eq (20), which implies that By Eq (22), similar to the proof of Lemma 2.1 in [33], we can deduce that there is a constant ϱ > 0 such that Using Eqs (27) and (10), and the mean-value theorem, we have where ξ 0 ∈ (0,1) and the last inequality follows Eq (15). Combining this result with Eq (26), for all sufficiently large k ∈ K 0, we obtain By Eq (27) and lim α = 0, the above inequality then implies that lim‖g ‖ = 0. This is also a contradiction. The proof is complete.

Lemma 0.3 Let Assumption (i) hold, and let the sequence {x } be generated by the RMPRP* method. Then, there are four positive numbers c , i = 1, 2, 3, 4 that satisfy

Proof. Considering the first inequality of Eq (28), we have where . Using we will discuss the three other inequalities of Eq (28). For , we have

Case 1: If ρ ≤ 0, then . Therefore, we have where and the second inequality follows from the mean-value theorem and Eq (29).

Case 2: If ρ > 0, then we have

Thus, we have

Let . We obtain and

Using the definition of , we obtain

Case 1: If ρ ≤ 0, we have

Case 2: If ρ > 0, we have and Letting , we obtain . The proof is complete.

Assumption (ii) In some neighborhood N of x*, ∇2 f is Lipschitz continuous.

Similar to [34], we can also establish the n-order quadratic convergence of the RMPRP* method. Here, we state the theorem but omit the proof.

Theorem 0.4 Let Assumptions (i) and (ii) hold. Then, there exists a constant c′ > 0 such that Specifically, the RMPRP* method is quadratically convergent.

Numerical Results

This section reports on various numerical experiments with Algorithm 4.1 and the normal algorithm to demonstrate the effectiveness of the given algorithm. We utilize the normal algorithm (called Algorithm N), whereby the algorithm does not use the restart technique, but the other steps are the same as those in Algorithm 4.1. We will test the following benchmark problems using these two algorithms. These problems are listed in Table 1. These benchmark problems and discussions concerning the choice of tested problems for an algorithm can be found at

Table 1

Definition of the benchmark problems and their features.

No.	Functions	Definition	Multimodal?	Separable?	Regular?
1	Sphere	f_Sph(x)=∑i=1pxi2 x i ∈ [−5.12, 5.12], x* = (0, 0, ⋯, 0), f Sph(x*) = 0.	no	yes	n/a
2	Schwefel’s	f_SchDS(x)=∑i=1p(∑j=1ix_j)^2 x i ∈ [−65.536, 65.536], x* = (0, 0, ⋯, 0), f SchDS(x*) = 0.	no	no	n/a
3	Rastrigin	f_Ras(x)=10p+∑i=1p(xi2−10cos(2πx_i)) x i ∈ [−5.12, 5.12], x* = (0, 0, ⋯, 0), f Ras(x*) = 0.	yes	yes	n/a
4	Schwefel	f_Sch(x)=418.9829p+∑i=1px_isin∣x_i∣ x i ∈ [−512.03, 511.97], x* = (−420.9678, −420.9678, ⋯, −420.9678), f Sch(x*) = 0.	yes	yes	n/a
5	Griewank	f_Gri(x)=1+∑i=1pxi24000−∏i=1pcosx_ii x i ∈ [−600, 600], x* = (0, 0, ⋯, 0), f Gri(x*) = 0.	yes	no	yes
6	Rosenbrock	f_Ros(x)=∑i=1p−1[100(x_i+1−xi2)^2+(x_i−1)^2] x i ∈ [−2.048, 2.048], x* = (1, ⋯, 1), f Ros(x*) = 0.	no	no	n/a
7	Ackley	f_Ack(x)=20+e−20e^−0.21p∑i=1pxi2−e^1p∑i=1pcos(2πx_i) x i ∈ [−30, 30], x* = (0, ⋯, 0), f Ack(x*) = 0.	yes	no	yes
8	Langerman	f_Lan(x)=−∑i=1mc_ie^−1π∑j=1p(x_j−a_ij)^2cos(π∑j=1p(x_j−a_ij)^2) x i ∈ [0, 10], m = p, x* = random, f Lan(x*) = random.	yes	no	no

We will use these benchmark problems to perform the experiments with these two algorithms. The code was written in MATLAB 7.6.0 and run on a PC with a Core 2 Duo CPU, E7500 @2.93 GHz, 2.00 GB memory and Windows XP operation system. The parameters of the algorithms are chosen to be ρ = 0.1, γ = 10, σ 1 = 0.0001, and γ = e 1 = ϵ = 10−4. The dimension can be found in Tables 2–3. Because the line search cannot always ensure that the descent condition holds, uphill searches will occur in the experiments. To avoid this situation, the step size α will be accepted if the searching number is larger than ten in every line search technique. The Himmeblau stop rule will be used:

Table 2

Test results using Algorithm 4.1.

	Dim	NI/NFG/f(x‾)	NI/NFG/f(x‾)	NI/NFG/f(x‾)	NI/NFG/f(x‾)
No.	x 0	(−5, ⋯)	(−3, ⋯)	(2, ⋯)	(4, ⋯)
1	30	2/6/8.355554e-023	2/6/3.274972e-024	2/6/1.455543e-024	2/6/5.822172e-024
	10000	2/6/2.788150e-020	2/6/1.091657e-021	2/6/4.851810e-022	2/6/1.940724e-021
	100000	2/6/5.970100e-019	2/6/8.813541e-020	2/6/4.871394e-021	2/6/1.948557e-020
	103000	2/6/5.962874e-019	2/6/9.164034e-020	2/6/4.997364e-021	2/6/1.998946e-020
	x 0	(−0.002, ⋯)	(−0.001, ⋯)	(0.001, ⋯)	(0.0002, ⋯)
2	30	4/12/7.338081e-006	4/12/1.834520e-006	4/12/1.834520e-006	3/9/4.070200e-007
	60	5/15/1.607584e-005	4/12/1.503191e-005	4/12/1.503191e-005	3/9/3.227252e-006
	100	6/18/2.574911e-005	5/15/1.889258e-005	5/15/1.889258e-005	4/12/2.785399e-006
	200	8/24/3.830680e-005	7/21/2.110989e-005	7/21/2.110989e-005	5/15/6.065297e-006
	x 0	(−0.02, 0, ⋯)	(0.01, 0, ⋯)	(0.001, 0, ⋯)	(0.006, 0, ⋯)
3	30	3/9/0.000000e+000	3/9/0.000000e+000	2/6/4.547474e-013	3/9/0.000000e+000
	60	3/9/0.000000e+000	3/9/0.000000e+000	2/6/1.136868e-012	3/9/0.000000e+000
	100	3/9/0.000000e+000	3/9/0.000000e+000	2/6/1.136868e-012	3/9/0.000000e+000
	200	3/9/0.000000e+000	3/9/0.000000e+000	2/6/1.136868e-012	3/9/0.000000e+000
	x 0	(−500, ⋯)	(−10, ⋯)	(−300, ⋯)	(50, ⋯)
4	30	9/28/4.588939e+002	5/15/1.233277e+004	10/30/8.751388e+003	2/16/1.469607e+004
	1000	9/28/1.529646e+004	5/15/4.110923e+005	10/30/2.917129e+005	2/16/4.898690e+005
	10000	9/28/1.529646e+005	5/15/4.110923e+006	10/30/2.917129e+006	2/16/4.898690e+006
	100000	9/28/1.529646e+006	5/15/4.110923e+007	10/30/2.917129e+007	2/16/4.898690e+007
	x 0	(−2, ⋯)	(−1, ⋯)	(1, ⋯)	(2, ⋯)
5	30	2/6/8.548717e-015	2/6/5.644248e-009	2/6/5.644248e-009	2/6/8.548717e-015
	1000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000
	10000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000
	100000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000
	x 0	(1.01, ⋯)	(0.9999, ⋯)	(1.003, ⋯)	(1.005, ⋯)
6	30	12/36/6.832422e-005	2/6/1.919312e-006	6/18/5.497859e-005	9/27/6.152662e-005
	1000	12/36/6.562848e-005	3/9/3.607120e-007	6/18/5.343543e-005	9/27/5.799691e-005
	10000	12/36/6.382394e-005	3/9/3.615221e-007	7/21/4.048983e-005	9/27/6.078016e-005
	100000	12/36/5.969884e-005	3/9/3.632823e-007	7/21/4.484866e-005	9/27/5.885821e-005
	x 0	(−0.1, ⋯)	(0.1, ⋯)	(0.01, ⋯)	(0.03, ⋯)
7	30	11/50/1.684545e+000	11/50/1.684545e+000	11/58/1.684467e+000	6/25/1.684388e+000
	1000	14/67/1.717428e+000	14/67/1.717428e+000	9/46/1.717374e+000	11/56/1.717448e+000
	10000	15/75/1.718189e+000	15/75/1.718189e+000	9/46/1.718252e+000	11/57/1.718327e+000
	100000	15/75/1.718273e+000	15/75/1.718273e+000	9/46/1.718340e+000	11/57/1.718410e+000
	x 0	(−5, ⋯)	(−1, ⋯)	(2, ⋯)	(4, ⋯)
8	30	1/3/-1.889333e-104	2/16/-8.226096e-004	1/3/-2.009161e-015	1/3/-1.228836e-064
	300	1/3/0.000000e+000	1/3/-3.893431e-040	1/3/-1.008253e-163	1/3/0.000000e+000
	500	1/3/0.000000e+000	1/3/-1.459270e-067	1/3/-4.297704e-274	1/3/0.000000e+000
	1000	1/3/0.000000e+000	1/3/-2.213361e-136	1/3/0.000000e+000	1/3/0.000000e+000

Table 3

Test results using Algorithm N.

	Dim	NI/NFG/f(x‾)	NI/NFG/f(x‾)	NI/NFG/f(x‾)	NI/NFG/f(x‾)
No.	x 0	(−5, ⋯)	(−3, ⋯)	(2, ⋯)	(4, ⋯)
1	30	2/6/8.355554e-023	2/6/3.274972e-024	2/6/1.455543e-024	2/6/5.822172e-024
	10000	2/6/2.788150e-020	2/6/1.091657e-021	2/6/4.851810e-022	2/6/1.940724e-021
	100000	2/6/5.970100e-019	2/6/8.813541e-020	2/6/4.871394e-021	2/6/1.948557e-020
	103000	2/6/5.962874e-019	2/6/9.164034e-020	2/6/4.997364e-021	2/6/1.998946e-020
	x 0	(−0.002, ⋯)	(−0.001, ⋯)	(0.001, ⋯)	(0.0002, ⋯)
2	30	4/12/7.338081e-006	4/12/1.834520e-006	4/12/1.834520e-006	3/9/4.070200e-007
	60	5/15/1.607584e-005	4/12/1.503191e-005	4/12/1.503191e-005	3/9/3.227252e-006
	100	6/18/2.574911e-005	5/15/1.889258e-005	5/15/1.889258e-005	4/12/2.785399e-006
	200	8/24/3.830680e-005	7/21/2.110989e-005	7/21/2.110989e-005	5/15/6.065297e-006
	x 0	(−0.02, 0, ⋯)	(0.01, 0, ⋯)	(0.001, 0, ⋯)	(0.006, 0, ⋯)
3	30	3/9/0.000000e+000	3/9/0.000000e+000	2/6/4.547474e-013	3/9/0.000000e+000
	60	3/9/0.000000e+000	3/9/0.000000e+000	2/6/1.136868e-012	3/9/0.000000e+000
	100	3/9/0.000000e+000	3/9/0.000000e+000	2/6/1.136868e-012	3/9/0.000000e+000
	200	3/9/0.000000e+000	3/9/0.000000e+000	2/6/1.136868e-012	3/9/0.000000e+000
	x 0	(−500, ⋯)	(−10, ⋯)	(−300, ⋯)	(50, ⋯)
4	30	9/28/4.588939e+002	5/15/1.233277e+004	10/30/8.751388e+003	2/16/1.469607e+004
	1000	9/28/1.529646e+004	5/15/4.110923e+005	10/30/2.917129e+005	2/16/4.898690e+005
	10000	9/28/1.529646e+005	5/15/4.110923e+006	10/30/2.917129e+006	2/16/4.898690e+006
	100000	9/28/1.529646e+006	5/15/4.110923e+007	10/30/2.917129e+007	2/16/4.898690e+007
	x 0	(−2, ⋯)	(−1, ⋯)	(1, ⋯)	(2, ⋯)
5	30	2/6/8.548717e-015	2/6/5.644248e-009	2/6/5.644248e-009	2/6/8.548717e-015
	1000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000
	10000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000
	100000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000	2/6/0.000000e+000
	x 0	(1.01, ⋯)	(0.9999, ⋯)	(1.003, ⋯)	(1.005, ⋯)
6	30	16/48/8.156782e-005	2/6/1.919312e-006	6/18/5.497859e-005	9/27/6.152662e-005
	1000	15/45/8.236410e-005	3/9/3.607120e-007	6/18/5.343543e-005	9/27/5.799691e-005
	10000	14/42/8.568194e-005	3/9/3.615221e-007	7/21/4.048983e-005	9/27/6.078016e-005
	100000	13/39/8.458113e-005	3/9/3.632823e-007	7/21/4.484866e-005	9/27/5.885821e-005
	x 0	(−0.1, ⋯)	(0.1, ⋯)	(0.01, ⋯)	(0.03, ⋯)
7	30	11/50/1.684545e+000	11/50/1.684545e+000	11/58/1.684467e+000	6/25/1.684388e+000
	1000	14/67/1.717428e+000	14/67/1.717428e+000	9/46/1.717374e+000	11/56/1.717448e+000
	10000	15/75/1.718189e+000	15/75/1.718189e+000	9/46/1.718252e+000	11/57/1.718327e+000
	100000	15/75/1.718273e+000	15/75/1.718273e+000	9/46/1.718340e+000	11/57/1.718410e+000
	x 0	(−5, ⋯)	(−1, ⋯)	(2, ⋯)	(4, ⋯)
8	30	1/3/-1.889333e-104	2/16/-8.226096e-004	1/3/-2.009161e-015	1/3/-1.228836e-064
	300	1/3/0.000000e+000	1/3/-3.893431e-040	1/3/-1.008253e-163	1/3/0.000000e+000
	500	1/3/0.000000e+000	1/3/-1.459270e-067	1/3/-4.297704e-274	1/3/0.000000e+000
	1000	1/3/0.000000e+000	1/3/-2.213361e-136	1/3/0.000000e+000	1/3/0.000000e+000

If ∣f(x )∣ > e 1, set otherwise, set stop1 = ∣f(x )−f(x )∣.

For each problem, the program will stop if ‖g(x)‖ < e 3 or stop1 < e 2 is satisfied, where e 1 = e 2 = e 3 = 10−4.

The program also stops if more than five thousand iterations are performed because the corresponding method for the problem is regarded as having failed. The meanings of the columns in Tables 2–3 are as follows:

x 0: the initial point NFG: the total number of NF and NG, i.e., NFG = NF+NG.

NI: the total number of iterations Dim: the dimension of the problem;

denotes the function value at the point when the program stops.

The results in Tables 2–3 show that these two algorithms effectively solve the benchmark problems—except for the fourth problem. In the experiment, the results when the algorithms are applied to problems 2 and 3 are not satisfactory when the dimensions of the problem are large; thus, we use a lower dimension. The dimensions of problems 8 and 9 are less than 1,000; the reasoning is similar to that for problems 2 and 3. However, the dimensions of all the problems are larger than 30, which is fixed. For many problems, the results are similar, except for those of problem 6. Clearly, the restart algorithm is competitive with the normal algorithm without using the restart technique.

To clearly to show the performance of Algorithm 4.1 and Algorithm N on NFG, we use the tool (S1 File) in [46] to analyze the algorithms. The results are listed in Table 4. It is easy to see that Algorithm 4.1 outperforms Algorithm N by approximately 1%, and we can conclude that the proposed method is better than the normal method. Thus, we hope that the given algorithm will be utilized in the future.

Table 4

The performance of Algorithm 4.1 and Algorithm N on NFG.

Algorithm 4.1	Algorithm N
0.99	1

Conclusion

This paper presented a new conjugate gradient method that exhibits global convergence, linear convergence, and quadratic convergence when suitable assumptions are made. Our proposed CG formula includes not only the gradient value information but also the function value information. For the test benchmark problems, the numerical results showed that the given algorithm is more effective than the normal method without using the restart technique.

We performed tests using benchmark problems using the presented algorithm and the normal algorithm without employing the restart technique. These two methods were shown to be very effective for solving the given problems. Moreover, we did not fix the dimension because n = = 30, and the largest dimension was higher than 100,000 (103,000). Additional numerical problems (such as the problems in [47]) should be investigated in the future to examine this algorithm.

Recently, we solved nonsmooth optimization problems using the relative gradient methods and obtained various results; therefore, in the future, we will use the RMPRP* method to solve nonsmooth optimization problems and hopefully obtain interesting results. Moreover, we will study the convergence of the CG method with other line search rules.

Supporting Information for Table 4.

(PDF)

Click here for additional data file.