Bounded perturbation resilience of extragradient-type methods and their applications.

In this paper we study the bounded perturbation resilience of the extragradient and the subgradient extragradient methods for solving a variational inequality (VI) problem in real Hilbert spaces. This is an important property of algorithms which guarantees the convergence of the scheme under summable errors, meaning that an inexact version of the methods can also be considered. Moreover, once an algorithm is proved to be bounded perturbation resilience, superiorization can be used, and this allows flexibility in choosing the bounded perturbations in order to obtain a superior solution, as well explained in the paper. We also discuss some inertial extragradient methods. Under mild and standard assumptions of monotonicity and Lipschitz continuity of the VI's associated mapping, convergence of the perturbed extragradient and subgradient extragradient methods is proved. In addition we show that the perturbed algorithms converge at the rate of [Formula: see text]. Numerical illustrations are given to demonstrate the performances of the algorithms.

Introduction

In this paper we are concerned with the variational inequality (VI) problem of finding a point such that where is a nonempty, closed and convex set in a real Hilbert space , denotes the inner product in , and is a given mapping. This problem is a fundamental problem in optimization theory, and it captures various applications such as partial differential equations, optimal control and mathematical programming; for the theory and application of VIs and related problems, the reader is referred, for example, to the works of Ceng et al. [1], Zegeye et al. [2], the papers of Yao et al. [3-5] and the many references therein.

Many algorithms for solving VI (1.1) are projection algorithms that employ projections onto the feasible set C of VI (1.1), or onto some related set, in order to reach iteratively a solution. Korpelevich [6] and Antipin [7] proposed an algorithm for solving (1.1), known as the extragradient method, see also Facchinei and Pang [8, Chapter 12]. In each iteration of the algorithm, in order to get the next iterate , two orthogonal projections onto C are calculated according to the following iterative step. Given the current iterate , calculate where , and L is the Lipschitz constant of F, or is updated by the following adaptive procedure: In the extragradient method there is the need to calculate twice the orthogonal projection onto C in each iteration. In case that the set C is simple enough so that projections onto it can be easily computed, this method is particularly useful; but if C is a general closed and convex set, a minimal distance problem has to be solved (twice) in order to obtain the next iterate. This might seriously affect the efficiency of the extragradient method. Hence, Censor et al. in [9-11] presented a method called the subgradient extragradient method, in which the second projection (1.2) onto C is replaced by a specific subgradient projection which can be easily calculated. The iterative step has the following form: where is the set defined as and .

In this manuscript we prove that the above methods, the extragradient and the subgradient extragradient methods, are bounded perturbation resilient, and the perturbed methods have the convergence rate of . This means that that will show that an inexact version of the algorithms allows incorporating summable errors that also converge to a solution of VI (1.1) and, moreover, their superiorized version can be introduced by choosing the perturbations. In order to obtain a superior solution with respect to some new objective function, for example, by choosing the norm, we can obtain a solution to VI (1.1) which is closer to the origin.

Our paper is organized as follows. In Section 2 we present the preliminaries. In Section 3 we study the convergence of the extragradient method with outer perturbations. Later, in Section 4, the bounded perturbation resilience of the extragradient method as well as the construction of the inertial extragradient methods are presented.

In the spirit of the previous sections, in Section 5 we study the convergence of the subgradient extragradient method with outer perturbations, show its bounded perturbation resilience and the construction of the inertial subgradient extragradient methods. Finally, in Section 6 we present numerical examples in signal processing which demonstrate the performances of the perturbed algorithms.

Preliminaries

Let be a real Hilbert space with the inner product and the induced norm , and let D be a nonempty, closed and convex subset of . We write to indicate that the sequence converges weakly to x and to indicate that the sequence converges strongly to x. Given a sequence , denote by its weak ω-limit set, that is, any such that there exists a subsequence of which converges weakly to x.

For each point , there exists a unique nearest point in D denoted by . That is, The mapping is called the metric projection of onto D. It is well known that is a nonexpansive mapping of onto D, i.e., and even firmly nonexpansive mapping. This is captured in the next lemma.

Lemma 2.1

For any and , it holds

;

.

The characterization of the metric projection [12, Section 3] is given by the following two properties in this lemma.

Lemma 2.2

Given and . Then if and only if and

Definition 2.3

The normal cone of D at denoted by is defined as

Definition 2.4

Let be a point-to-set operator defined on a real Hilbert space . The operator B is called a maximal monotone operator if B is monotone, i.e., and the graph of B, is not properly contained in the graph of any other monotone operator.

Based on Rockafellar [13, Theorem 3], a monotone mapping B is maximal if and only if, for any , if for all , then it follows that .

Definition 2.5

The subdifferential set of a convex function c at a point x is defined as

For , take any and define This is a half-space, the bounding hyperplane of which separates the set D from the point z if ; otherwise ; see, e.g., [14, Lemma 7.3].

Lemma 2.6

([15])

Let D be a nonempty, closed and convex subset of a Hilbert space . Let be a bounded sequence which satisfies the following properties: Then converges to a point in D.

every limit point of lies in D;

exists for every .

Lemma 2.7

Assume that is a sequence of nonnegative real numbers such that where the nonnegative sequences and satisfy and , respectively. Then exists.

The extragradient method with outer perturbations

In order to discuss the convergence of the extragradient method with outer perturbations, we make the following assumptions.

Condition 3.1

The solution set of (1.1), denoted by , is nonempty.

Condition 3.2

The mapping F is monotone on C, i.e.,

Condition 3.3

The mapping F is Lipschitz continuous on C with the Lipschitz constant , i.e.,

Observe that while Censor et al. in [10, Theorem 3.1] showed the weak convergence of the extragradient method (1.2) in Hilbert spaces for a fixed step size , this can be easily improved in case that the adaptive rule (1.3) is used. The next theorem shows this, and its proof can easily be derived by following similar lines of the proof of [10, Theorem 3.1].

Theorem 3.4

Assume that Conditions 3.1-3.3 hold. Then any sequence generated by the extragradient method (1.2) with the adaptive rule (1.3) weakly converges to a solution of the variational inequality (1.1).

Denote , . The sequences of perturbations , , are assumed to be summable, i.e.,

Now we consider the extragradient method with outer perturbations.

Algorithm 3.5

The extragradient method with outer perturbations:

Step 0: Select a starting point and set .

Step 1: Given the current iterate , compute where , , and is the smallest nonnegative integer such that (see [16]) Calculate the next iterate

Step 2: If , then stop. Otherwise, set and return to Step 1.

Convergence analysis

Lemma 3.6

([17])

The Armijo-like search rule (3.5) is well defined. Besides, , where .

Theorem 3.7

Assume that Conditions 3.1-3.3 hold. Then the sequence generated by Algorithm 3.5 converges weakly to a solution of the variational inequality (1.1).

Proof

Take . From (3.6) and Lemma 2.1(ii), we have From the Cauchy-Schwarz inequality and the mean value inequality, it follows Using and the monotone property of F, we have and, consequently, we get Thus, we have where the equality comes from Using , the definition of and Lemma 2.2, we have So, we obtain From (3.3), it follows Therefor, we assume and , , where . So, using (3.13), we get Combining (3.7)-(3.10) and (3.15), we obtain where From (3.16), it follows Since , , we get So, from (3.18), we have Using (3.3) and Lemma 2.7, we get the existence of and then the boundedness of . From (3.20), it follows which means that Thus, we obtain and consequently,

Now, we are to show . Due to the boundedness of , it has at least one weak accumulation point. Let . Then there exists a subsequence of which converges weakly to x̂. From (3.23), it follows that also converges weakly to x̂.

We will show that x̂ is a solution of the variational inequality (1.1). Let where is the normal cone of C at . It is known that A is a maximal monotone operator and . If , then we have since . Thus it follows that Since , we have

On the other hand, by the definition of and Lemma 2.2, it follows that and consequently, Hence we have which implies Taking the limit as in the above inequality and using Lemma 3.6, we obtain Since A is a maximal monotone operator, it follows that . So, .

Since exists and , using Lemma 2.6, we conclude that weakly converges to a solution of the variational inequality (1.1). This completes the proof. □

Convergence rate

Nemirovski [18] and Tseng [19] proved the convergence rate of the extragradient method. In this subsection, we present the convergence rate of Algorithm 3.5.

Theorem 3.8

Assume that Conditions 3.1-3.3 hold. Let the sequences and be generated by Algorithm 3.5. For any integer , we have which satisfies where and

Take arbitrarily . From Conditions 3.2 and 3.3, we have By (3.6) and Lemma 2.2, we get Identifying with x in (3.7) and (3.8), and combining (3.36) and (3.37), we get Thus, we have where . Summing inequality (3.39) over , we obtain Using the notations of and in the above inequality, we derive The proof is complete. □

Remark 3.9

From Lemma 3.6, it follows thus Algorithm 3.5 has convergence rate. In fact, for any bounded subset and given accuracy , our algorithm achieves in at most iterations, where is defined by (3.34) and .

The bounded perturbation resilience of the extragradient method

In this section, we prove the bounded perturbation resilience (BPR) of the extragradient method. This property is fundamental for the application of the superiorization methodology (SM) to them.

The superiorization methodology of [20-22], which originates in the papers by Butnariu, Reich and Zaslavski [23-25], is intended for constrained minimization (CM) problems of the form where is an objective function and is the solution set of another problem. Here, we assume throughout this paper. Assume that the set Ψ is a closed convex subset of a Hilbert space H, the minimization problem (4.1) becomes a standard CM problem. Here, we are interested in the case wherein Ψ is the solution set of another CM of the form i.e., we wish to look at provided that Ψ is nonempty. If f is differentiable, and let , then CM (4.2) equals the following variational inequality: to find a point such that

The superiorization methodology (SM) strives not to solve (4.1) but rather the task is to find a point in Ψ which is superior, i.e., has a lower, but not necessarily minimal, value of the objective function ϕ. This is done in the SM by first investigating the bounded perturbation resilience of an algorithm designed to solve (4.2) and then proactively using such permitted perturbations in order to steer the iterates of such an algorithm toward lower values of the ϕ objective function while not loosing the overall convergence to a point in Ψ.

In this paper, we do not investigate superiorization of the extragradient method. We prepare for such an application by proving the bounded perturbation resilience that is needed in order to do superiorization.

Algorithm 4.1

The basic algorithm:

Initialization: is arbitrary;

Iterative step: Given the current iterate vector , calculate the next iterate via

The bounded perturbation resilience (henceforth abbreviated to BPR) of such a basic algorithm is defined next.

Definition 4.2

An algorithmic operator is said to be bounded perturbations resilient if the following is true. If Algorithm 4.5 generates sequences with that converge to points in Ψ, then any sequence starting from any , generated by also converges to a point in Ψ, provided that (i) the sequence is bounded, and (ii) the scalars are such that for all , and , and (iii) for all .

Definition 4.2 is nontrivial only if , in which condition (iii) is enforced in the superiorized version of the basic algorithm, see step (xiv) in the ‘Superiorized Version of Algorithm P’ in ([26], p.5537) and step (14) in ‘Superiorized Version of the ML-EM Algorithm’ in ([27], Subsection II.B). This will be the case in the present work.

Treating the extragradient method as the Basic Algorithm , our strategy is to first prove convergence of the iterative step (1.2) with bounded perturbations. We show next how the convergence of this yields BPR according to Definition 4.2.

A superiorized version of any Basic Algorithm employs the perturbed version of the Basic Algorithm as in (4.6). A certificate to do so in the superiorization method, see [28], is gained by showing that the Basic Algorithm is BPR. Therefore, proving the BPR of an algorithm is the first step toward superiorizing it. This is done for the extragradient method in the next subsection.

The BPR of the extragradient method

In this subsection, we investigate the bounded perturbation resilience of the extragradient method whose iterative step is given by (1.2).

To this end, we treat the right-hand side of (1.2) as the algorithmic operator of Definition 4.2, namely, we define, for all , and identify the solution set Ψ with the solution set of the variational inequality (1.1) and identify the additional set Θ with C.

According to Definition 4.2, we need to show the convergence of the sequence that, starting from any , is generated by which can be rewritten as where , , and is the smallest nonnegative integer such that The sequences and obey conditions (i) and (ii) in Definition 4.2, respectively, and also (iii) in Definition 4.2 is satisfied.

The next theorem establishes the bounded perturbation resilience of the extragradient method. The proof idea is to build a relationship between BPR and the convergence of the iterative step (1.2).

Theorem 4.3

Assume that Conditions 3.1-3.3 hold. Assume the sequence is bounded, and the scalars are such that for all , and . Then the sequence generated by (4.9) and (4.10) converges weakly to a solution of the variational inequality (1.1).

Take . From and that is bounded, we have which means So, we assume , where . Identifying with in (3.7) and (3.8) and using (3.10), we get

From , the definition of and Lemma 2.2, we have So, we obtain We have Similar to (3.8), we can show Combining (4.14)-(4.16), we get where the last inequality comes from and . Substituting (4.17) into (4.13), we get Following the proof line of Theorem 3.7, we get weakly converges to a solution of the variational equality (1.1). □

By using Theorems 3.8 and 4.3, we obtain the convergence rate of the extragradient method with BP.

Theorem 4.4

Assume that Conditions 3.1-3.3 hold. Assume the sequence is bounded, and the scalars are such that for all , and . Let the sequences and be generated by (4.9) and (4.10). For any integer , we have which satisfies where and

Construction of the inertial extragradient methods by BPR

In this subsection, we construct two classes of inertial extragradient methods by using BPR, i.e., identifying , , and , with special values.

Polyak [29, 30] first introduced the inertial-type algorithms by using the heavy ball method of the second-order dynamical systems in time. Since the inertial-type algorithms speed up the original algorithms without the inertial effects, recently there has been increasing interest in studying inertial-type algorithms (see, e.g., [31-34]). The authors [35] introduced an inertial extragradient method as follows: for each , where , is nondecreasing with and for each and are such that and where L is the Lipschitz constant of F.

Based on the iterative step (1.2), we construct the following inertial extragradient method: where

Theorem 4.5

Assume that Conditions 3.1-3.3 hold. Assume that the sequences , , satisfy , . Then the sequence generated by the inertial extragradient method (4.24) converges weakly to a solution of the variational inequality (1.1).

Let , , where It is obvious that . So, it follows that , , satisfy (3.3) from the condition on . Using Theorem 3.7, we complete the proof. □

Remark 4.6

From (3.24), we have for big enough k, that is, .

Using the extragradient method with bounded perturbations (4.9), we construct the following inertial extragradient method: where We extend Theorem 4.3 to the convergence of the inertial extragradient method (4.27).

Theorem 4.7

Assume that Conditions 3.1-3.3 hold. Assume that the sequence satisfies . Then the sequence generated by the inertial extragradient method (4.27) converges weakly to a solution of the variational inequality (1.1).

Remark 4.8

The inertial parameter in the inertial extragradient method (4.24) is bigger than that of the inertial extragradient method (4.27). The inertial extragradient method (4.24) becomes the inertial extragradient method (4.27) when .

The extension to the subgradient extragradient method

In this section, we generalize the results of extragradient method proposed in the previous sections to the subgradient extragradient method.

Censor et al. [9] presented the subgradient extragradient method (1.4). In their method the step size is fixed , where L is a Lipschitz constant of F. So, in order to determine the stepsize , one needs first calculate (or estimate) L, which might be difficult or even impossible in general. So, in order to overcome this, the Armijo-like search rule can be used:

To discuss the convergence of the subgradient extragradient method, we make the following assumptions.

Condition 5.1

The mapping F is monotone on , i.e.,

Condition 5.2

The mapping F is Lipschitz continuous on with the Lipschitz constant , i.e.,

As before, Censor et al.’s subgradient extragradient method [10, Theorem 3.1] can be easily generalized by using some adaptive step rule, for example, (5.1). This result is captured in the next theorem.

Theorem 5.3

Assume that Conditions 3.1, 5.1 and 5.2 hold. Then the sequence generated by the subgradient extragradient methods (1.4) and (5.1) weakly converges to a solution of the variational inequality (1.1).

The subgradient extragradient method with outer perturbations

In this subsection, we present the subgradient extragradient method with outer perturbations.

Algorithm 5.4

The subgradient extragradient method with outer perturbations:

Step 0: Select a starting point and set .

Step 1: Given the current iterate , compute where , , and is the smallest nonnegative integer such that (see [16]) Construct the set and calculate

Step 2: If , then stop. Otherwise, set and return to Step 1.

Denote , . The sequences of perturbations , , are assumed to be summable, i.e.,

Following the proof of Theorems 3.7 and 3.8, we get the convergence analysis and convergence rate of Algorithm 5.4.

Theorem 5.5

Assume that Conditions 3.1, 5.1 and 5.2 hold. Then the sequence generated by Algorithm 5.4 converges weakly to a solution of the variational inequality (1.1).

Theorem 5.6

Assume that Conditions 3.1, 5.1 and 5.2 hold. Let the sequences and be generated by Algorithm 5.4. For any integer , we have which satisfies where and

The BPR of the subgradient extragradient method

In this subsection, we investigate the bounded perturbation resilience of the subgradient extragradient method (1.4).

To this end, we treat the right-hand side of (1.4) as the algorithmic operator of Definition 4.2, namely, we define, for all , where satisfies (5.1) and Identify the solution set Ψ with the solution set of the variational inequality (1.1) and identify the additional set Θ with C.

According to Definition 4.2, we need to show the convergence of the sequence that, starting from any , is generated by which can be rewritten as where , , and is the smallest nonnegative integer such that The sequences and obey conditions (i) and (ii) in Definition 4.2, respectively, and also (iii) in Definition 4.2 is satisfied.

The next theorem establishes the bounded perturbation resilience of the subgradient extragradient method. Since its proof is similar to that of Theorem 4.3, we omit it.

Theorem 5.7

Assume that Conditions 3.1, 5.1 and 5.2 hold. Assume the sequence is bounded, and the scalars are such that for all , and . Then the sequence generated by (5.15) and (5.16) converges weakly to a solution of the variational inequality (1.1).

We also get the convergence rate of the subgradient extragradient methods with BP (5.15) and (5.16).

Theorem 5.8

Assume that Conditions 3.1, 5.1 and 5.2 hold. Assume the sequence is bounded, and the scalars are such that for all , and . Let the sequences and be generated by (5.15) and (5.16). For any integer , we have which satisfies where and

Construction of the inertial subgradient extragradient methods by BPR

In this subsection, we construct two classes of inertial subgradient extragradient methods by using BPR, i.e., identifying , , and , with special values.

Based on Algorithm 5.4, we construct the following inertial subgradient extragradient method: where satisfies (5.16) and

Similar to the proof of Theorem 4.5, we get the convergence of the inertial subgradient extragradient method (5.20).

Theorem 5.9

Assume that Conditions 3.1, 5.1 and 5.2 hold. Assume that the sequences , , satisfy , . Then the sequence generated by the inertial subgradient extragradient method (5.20) converges weakly to a solution of the variational inequality (1.1).

Using the subgradient extragradient method with bounded perturbations (5.15), we construct the following inertial subgradient extragradient method: where , , and is the smallest nonnegative integer such that and We extend Theorem 4.3 to the convergence of the inertial subgradient extragradient method (5.22).

Theorem 5.10

Assume that Conditions 3.1, 5.1 and 5.2 hold. Assume that the sequence satisfies . Then the sequence generated by the inertial subgradient extragradient method (5.22) converges weakly to a solution of the variational inequality (1.1).

Numerical experiments

In this section, we provide three examples to compare the inertial extragradient method (4.22) (iEG1), the inertial extragradient method (4.24) (iEG2), the inertial extragradient method (4.27) (iEG), the extragradient method (1.2), the inertial subgradient extragradient method (5.20) (iSEG1), the inertial subgradient extragradient method (5.22) (iSEG2) and the subgradient extragradient method (1.4).

In the first example, we consider a typical sparse signal recovery problem. We choose the following set of parameters. Take , and . Set in the inertial extragradient methods (4.22) and (4.24), and the inertial subgradient extragradient methods (5.20) and (5.22). Choose and in the inertial extragradient method (4.24).

Example 6.1

Let be a K-sparse signal, . The sampling matrix () is stimulated by the standard Gaussian distribution and a vector , where e is additive noise. When , it means that there is no noise to the observed data. Our task is to recover the signal from the data b.

It is well known that the sparse signal can be recovered by solving the following LASSO problem [36]: where . It is easy to see that the optimization problem (6.1) is a special case of the variational inequality problem (1.1), where and . We can use the proposed iterative algorithms to solve the optimization problem (6.1). Although the orthogonal projection onto the closed convex set C does not have a closed-form solution, the projection operator can be precisely computed in a polynomial time. We include the details of computing in Appendix. We conduct plenty of simulations to compare the performances of the proposed iterative algorithms. The following inequality was defined as the stopping criterion: where is a given small constant. ‘Iter’ denotes the iteration numbers. ‘Obj’ represents the objective function value and ‘Err’ is the 2-norm error between the recovered signal and the true K-sparse signal. We divide the experiments into two parts. One task is to recover the sparse signal from noise observation vector b, and the other is to recover the sparse signal from noiseless data b. For the noiseless case, the obtained numerical results are reported in Table 1. To visually view the results, Figure 1 shows the recovered signal compared with the true signal when . We can see from Figure 1 that the recovered signal is the same as the true signal. Further, Figure 2 presents the objective function value versus the iteration numbers.

Table 1

Numerical results obtained by the proposed iterative algorithms when , in the noiseless case

K -sparse signal	Methods	ϵ=10^−4			ϵ=10^−6
		Iter	Obj	Err	Iter	Obj	Err
K = 20	EG	444	9.7346e-4	0.0080	817	9.6625e-8	7.9856e-5
	SEG	444	9.7272e-4	0.0080	817	9.6555e-8	7.9827e-5
	iEG	374	6.2389e-4	0.0064	675	6.3456e-8	6.4715e-5
	iEG1	159	7.0799e-5	0.0021	263	7.4280e-9	2.2041e-5
	iEG2	158	8.3897e-5	0.0023	273	1.0889e-8	2.6809e-5
	iSEG1	415	8.9563e-4	0.0076	787	2.3571e-7	5.2470e-5
	iSEG2	414	9.2167e-4	0.0077	760	9.1586e-8	7.7275e-5
K = 30	EG	1,285	0.0035	0.0281	2,583	3.4535e-7	2.8035e-4
	SEG	1,285	0.0035	0.0281	2,583	3.4534e-7	2.8035e-4
	iEG	1,091	0.0023	0.0227	2,144	2.2732e-7	2.2745e-4
	iEG1	532	3.7493e-4	0.0092	944	3.7522e-8	9.2287e-5
	iEG2	535	3.7961e-4	0.0093	956	4.3181e-8	9.3120e-5
	iSEG1	1,176	0.0031	0.0266	2,351	3.1038e-7	2.6137e-4
	iSEG2	1,176	0.0031	0.0266	2,346	3.1635e-7	2.6784e-4
K = 40	EG	1,729	0.0050	0.0405	3,599	5.0237e-7	4.0488e-4
	SEG	1,729	0.0050	0.0405	3,599	5.0228e-7	4.0484e-4
	iEG	1,473	0.0033	0.0328	2,990	3.3182e-7	3.2905e-4
	iEG1	744	5.4838e-4	0.0134	1,361	5.5456e-8	1.3440e-4
	iEG2	745	5.4807e-4	0.0134	1,355	6.4785e-8	1.4191e-4
	iSEG1	1,570	0.0045	0.0384	3,246	4.5079e-7	3.8146e-4
	iSEG2	1,572	0.0045	0.0382	3,244	4.5389e-7	3.8435e-4

Figure 1

Comparison of the different methods for sparse signal recovery. (a1) is the true sparse signal, (a2)-(a8) are the recovered signals vs the true signal by ‘EG’, ‘SEG’, ‘iEG’, ‘iEG1’, ‘iEG2’, ‘iSEG1’ and ‘iSEG2’, respectively.

Figure 2

Comparison of the objective function value versus the iteration numbers of different methods.

For the noise observation b, we assume that the vector e is corrupted by Gaussian noise with zero mean and β variances. The system matrix A is the same as in the noiseless case, and the sparsity level . We list the numerical results for different noise level β in Table 2. When the noise , Figure 3 shows the objective function value versus the iteration numbers. Figure 4 shows the recovered signal vs the true signal in the noise case.

Table 2

Numerical results for the proposed iterative algorithms with different noise value

Variances	Methods	ϵ=10^−4			ϵ=10^−6
		Iter	Obj	Err	Iter	Obj	Err
β = 0.01	EG	1,264	0.0092	0.0317	2,192	0.0061	0.0131
	SEG	1,264	0.0092	0.0317	2,192	0.0061	0.0131
	iEG	1,070	0.0081	0.0272	1,812	0.0061	0.0131
	iEG1	519	0.0063	0.0164	788	0.0061	0.0130
	iEG2	516	0.0063	0.0166	786	0.0061	0.0130
	iSEG1	1,156	0.0089	0.0305	1,995	0.0061	0.0131
	iSEG2	1,157	0.0089	0.0304	1,990	0.0061	0.0131
β = 0.02	EG	1,274	0.0163	0.0387	2,086	0.0142	0.0272
	SEG	1,274	0.0163	0.0387	2,086	0.0142	0.0272
	iEG	1,070	0.0154	0.0356	1,728	0.0142	0.0272
	iEG1	492	0.0144	0.0300	756	0.0142	0.0272
	iEG2	495	0.0143	0.0300	759	0.0142	0.0272
	iSEG1	1,163	0.0161	0.0378	1,899	0.0142	0.0272
	iSEG2	1,161	0.0161	0.0380	1,895	0.0142	0.0272
β = 0.05	EG	1,190	0.1012	0.0749	1,869	0.0991	0.0651
	SEG	1,190	0.1012	0.0749	1,869	0.0991	0.0651
	iEG	996	0.1005	0.0727	1,542	0.0991	0.0650
	iEG1	460	0.0993	0.0677	670	0.0991	0.0650
	iEG2	461	0.0993	0.0675	665	0.0991	0.0650
	iSEG1	1,084	0.1010	0.0742	1,704	0.0991	0.0651
	iSEG2	1,084	0.1010	0.0742	1,704	0.0991	0.0651

Figure 3

Comparison of the objective function value versus the iteration numbers of different methods in the noise case of .

Figure 4

Comparison of the different methods for sparse signal recovery. (a1) is the true sparse signal, (a2)-(a8) are the recovered signals vs the true signal by ‘EG’, ‘SEG’, ‘iEG’, ‘iEG1’, ‘iEG2’, ‘iSEG1’ and ‘iSEG2’ in the noise case of , respectively.

Example 6.2

Let be defined by

The authors [37] proved that F is Lipschitz continuous with and 1-strongly monotone. Therefore the variational inequality (1.1) has a unique solution, and is its solution.

Let , where and . Take the initial point . Since is the unique solution of the variational inequality (1.1), denote by the stopping criterion.

Example 6.3

Let defined by , where , and , where and are generated randomly.

It is easy to verify that F is L-Lipschitz continuous and η-strongly monotone with and .

Let , where the center and radius are randomly chosen. Take the initial point , where is generated randomly. Set . Take and other parameters are set the same values as in Example 6.2. Although the variational inequality (1.1) has a unique solution, it is difficult to get the exact solution. So, denote by the stopping criterion.

From Figures 5 and 6, we conclude: (i) the inertial-type algorithms improve the original algorithms; (ii) the performances of the inertial extragradient methods (4.22) and (4.24) are almost the same; (iii) the inertial subgradient extragradient method (5.20) performs better than the inertial subgradient extragradient method (5.22) for Example 6.1, while they are almost the same for Example 6.2; (iv) the (inertial) extragradient methods behave better than the (inertial) subgradient extragradient methods since the sets C in Examples 6.2 and 6.3 are simple, and hence the computation load of the projection onto it is small; (v) the inertial extragradient method (4.22) has an advantage over the inertial extragradient methods (4.22) and (4.24). The reason may be that it takes a bigger inertial parameter .

Figure 5

Comparison of the number of iterations of different methods for Example .

Figure 6

Comparison of the number of iterations of different methods for Example .

Conclusions

In this research article we study an important property of iterative algorithms for solving variational inequality (VI) problems which is called bounded perturbation resilience. In particular, we focus on extragradient-type methods. This enables us to develop inexact versions of the methods as well as apply the superiorization methodology in order to obtain a ‘superior’ solution to the original problem. In addition, some inertial extragradient methods are also derived. All the presented methods converge at the rate of , and three numerical examples illustrate, demonstrate and compare the performances of all the algorithms.