New Results on Passivity Analysis of Stochastic Neural Networks with Time-Varying Delay and Leakage Delay.

The passivity problem for a class of stochastic neural networks systems (SNNs) with varying delay and leakage delay has been further studied in this paper. By constructing a more effective Lyapunov functional, employing the free-weighting matrix approach, and combining with integral inequality technic and stochastic analysis theory, the delay-dependent conditions have been proposed such that SNNs are asymptotically stable with guaranteed performance. The time-varying delay is divided into several subintervals and two adjustable parameters are introduced; more information about time delay is utilised and less conservative results have been obtained. Examples are provided to illustrate the less conservatism of the proposed method and simulations are given to show the impact of leakage delay on stability of SNNs.

1. Introduction

During the past several decades, neural networks have gained great attention because of their potential application in pattern classification, reconstruction of moving image, and combinatorial optimization. In addition, time delay is a natural phenomenon frequently encountered in various dynamic systems such as electronic, chemical systems, long transmission lines in pneumatic systems, biological systems, and economic and rolling mill systems. Delays in neural networks can cause oscillation, instability, and divergence, which are very often the main sources of poor performance of designed neural networks. So the stability analysis and state estimation of neural networks with various time delays have been widely investigated by many researchers; see [1-8] and the references therein.

Furthermore, when modeling real nervous systems, stochastic disturbance is one of main resources of the performance degradations when applying the neural networks, because the synaptic transmission is a noisy process introduced by random fluctuation from the release of neurotransmitter and other probabilistic causes. In recent years, the stability analysis for stochastic neural networks with time delay has become a hot research topic; by virtue of various inequality technics and M-matrix theory, many important research results about neural networks with different type of time delays, such as constant delay, time-varying delay, or distributed delay, have been reported; see, for example, [8-14] and the references therein.

The passivity theory, which originated from circuit theory, plays an important role in the analysis of stability of linear or nonlinear systems. The main character of passivity theory is that the passive properties of a system can keep the system internally stable. Because it is a very effective tool in studying the stability of uncertain or nonlinear systems, the passivity theory has been used widely in fuzzy control [15], complexity [16], synchronization [17], signal processing [18], and adaptive control [19].

Recently, based on the Lyapunov-Krasovskii theory, passivity and dissipativity analysis of neural networks with various delays and uncertainties have been discussed and many interesting results have been reported [20-28].

In [29-35], based on the Lyapunov-Krasovskii, LMI method, and a delay fractioning technique, the passivity and robust passivity of stochastic neural networks with delays and uncertainties have been studied; some sufficient conditions on the passivity of neural networks with delays have been obtained. In [31], authors investigated passivity of the stochastic neural networks with time-varying delays and parameters uncertainties by applying free-weighting matrix and the lower conservatism results are obtained by comparing with the existing results.

On the other hand, in many practical problems, a typical time delay called leakage delay or forgetting delay exists in dynamical system, which has a tendency to destabilize the system; it has been one of the research hot topics recently and many research achievements have been reported [20, 36–42].

As pointed out in [36], neural networks with leakage delay are a class of important neural networks, and time delay in the leakage term also has great impact on the dynamics of neural networks; sometimes it has more significant effect than other kinds of delays on dynamics of neural networks; the stability analysis of neural networks system involving leakage delay has been researched extensively; see, for example, [37-40] and the references therein. Very recently, in [42], by virtue of free weight matrix and LMIs method, the passivity problem for a class of stochastic neural networks with leakage delay is studied; the sufficient condition making the system passive is presented, but leakage delay under consideration is a constant; but, in practical dynamical systems, the leakage delay can be time-varying, which is often more general and complex than leakage delay being a constant. To the best of authors' knowledge, no research results have been reported about the condition that leakage delay is time-varying, which motivates our idea.

Motivated by the aforementioned discussions, this paper focuses on the passivity problem for a class of stochastic neural networks (SNNs) system with time-varying delay and leakage delay; by constructing a new Lyapunov functional, a set of sufficient conditions are derived to ensure the passivity performance for a class of stochastic neural networks with time-varying delays and leakage delay. By virtue of the delay decomposition idea [8], combining with some integral inequality technic [7], or free-weighting matrix approach [9, 26], two adjustable parameters are introduced and made full use of. All results are established in the form of LMIs and can be solved easily by using the interior algorithms, which can be efficiently solved by Matlab LMI Toolbox and no tuning of parameters is required. Finally, numerical examples are given to demonstrate the effectiveness and less conservatism of the proposed approach.

The main contributions of this paper are summarized as follows:

The leakage delay studied is time-varying, so the research model is more general and complex than that in [42].

The neuron activation function is assumed to satisfy sector-bounded condition, which is more general and less restrictive than Lipschitz condition, so the less conservatism results can be expected.

The derivative of time-varying can be extended to be more than 1.

How the leakage delay affects the stability result is discussed.

Notation. Throughout this paper, if not explicit, matrices are assumed to have compatible dimensions. The notation M > (≥, <, ≤)  0 means that the symmetric matrix M is positive-definite (positive-semidefinite, negative, and negative-semidefinite). λ min(·) and λ max(·) denote the minimum and the maximum eigenvalue of the corresponding matrix; the superscript “T” stands for the transpose of a matrix; the shorthand diag{⋯} denotes the block diagonal matrix; ‖·‖ represents the Euclidean norm for vector or the spectral norm of matrices. I refers to an identity matrix of appropriate dimensions. 𝔼{·} stands for the mathematical expectation; ∗ means the symmetric terms. Sometimes, the arguments of a function will be omitted in the analysis when no confusion can arise.

2. System Description

Consider the SNNs with time-varying delay as follows:where x(t) = [x 1(t), x 2(t),…, x (t)] ∈ ℝ is the neural state vector and u(t) = [u 1(t), u 2(t),…, u (t)] is the input. y(t) = [y 1(t), y 2(t),…, y (t)] ∈ ℝ is the output; W 0, W 1 ∈ ℝ are the connection weight matrix and the delayed connection weight matrix, respectively; A = diag⁡(a 1, a 2,…, a ) is a positive diagonal matrix; f(x(t)) = [f 1(x(t)), f 2(x(t)),…, f (x(t))] ∈ ℝ is the neuron activation function with f(0) = 0; n denotes the number of neurons in neural networks; ω(t) = [ω 1(t), ω 2(t),…, ω (t)] ∈ ℝ is an m-dimension Brownian motion defined on a complete probability space (Ω, ℱ, P), satisfying τ(t) is the transmission delay and is assumed to satisfy δ(t) is the leakage delay that satisfieswhere τ, μ, δ, ρ are some positive scalar constants.

Assumption 1 .

For i ∈ {1,2,…, n} and ∀x, y ∈ ℝ, x ≠ y, the neuron activation function f(·) is continuous and bounded and satisfieswhere Λ1 and Λ2 are some constant known matrices.

Remark 2 .

In this paper, the above assumption is made on neuron activation function, which is called sector-bounded neuron activation function. When Λ1 = Λ2 = −Λ, condition (5) becomesSo it is less restrictive than the descriptions on both the sigmoid activation functions and the Lipschitz-type activation functions.

Assumption 3 .

There exist three constant matrices Σ1, Σ2, and Σ3 such that

Definition 4 (see [22]).

The delayed SNNs are said to stochastically passive if there exists a scalar γ ≥ 0 such thatfor all t ≥ 0 and for all solution of (1) with x(0) = 0.

Remark 5 .

The different output equation can lead to different definitions. In [31, 42], the output equation expression is y(t) = f(x(t)) and y(t) = Gf(x(t)), respectively. In order to compare our result with that in [42], we take G = I, so we have the same definition as that in [42].

At first, we give the following lemmas which will be used frequently in the proof of the our main results.

Lemma 6 (see [4]).

For any constant symmetric positive defined matrix J ∈ ℝ , scalar η, and the vector function ν : [0, η] → ℝ , the following inequality holds:

Lemma 7 (see [5]).

For given proper dimensions constant matrices Φ1, Φ2, and Φ3, where Φ1 = Φ1 and Φ2 = Φ2 > 0, we have Φ1 + Φ3 Φ2 −1Φ3 < 0 such that only and only if

Lemma 8 (see [7]).

For given function τ(t) satisfying , there exist nonnegative functions λ 1(t) ≥ 0 and λ 2(t) ≥ 0 satisfying λ 1(t) + λ 2(t) = 1 such that the following equation holds:

Lemma 9 (see [7]).

For any real vectors a and b and any matrix Q > 0 with appropriate dimensions, it follows that ±2a b ≤ a Qa + b Q −1 b.

3. Main Results

In this section, a delay-dependent leakage delay method is developed to guarantee the stochastic passive results of system (1), so we have the following Theorem 10.

Theorem 10 .

Given scalars τ > 0, 0 < α < 1, 0 < β < 1, λ > 0, ρ > 0, and 0 < μ and proper matrix Σ  (i = 1,2, 3), the SNNs described by (1) are stochastically passive in the sense of Definition 4, if there exist positive matrices P > 0, Q > 0, Q > 0  (j = 1,2,…, 5), and R > 0  (l = 1,2,…, 5), positive diagonal matrices F > 0  (j = 1,2), positive constants ϵ 1, ϵ 2, γ > 0, and real matrices , , and of appropriate dimensions such that the following LMIs hold: where Ψ1 and Ψ2 are defined as replacing in Ψ by μ and −μ, respectively. Consider

Proof

For the convenience of proof, we denotethen system (1) can be rewritten as

Choose a Lyapunov-Krasovskii functional candidate as V(x(t)) = ∑ 6 V (x(t), t), wherewhere φ(t) = τ(t) + β(τ − τ(t)) and 0 < α < 1 and 0 < β < 1. Then, the stochastic differential of V(x(t), t) along system (1) can be obtained as follows:where

So by Lemma 9, the following inequalities can be obtained:where ,By Lemma 6, it is easy to know thatFor arbitrary matrices , , , , and with compatible dimensions, we have wherewhereFrom Assumptions 3 and (12), we can get

In addition, from Assumption 1, the following inequalities can be deduced:

It is clear that for any scalars ϵ 1 > 0 and ϵ 2 > 0, there exist diagonal matrices F 1 ≥ 0, F 2 ≥ 0, and Λ  (i = 1,2) such that the following inequality hold:where

In order to get the passive condition, we introduce the following inequality:On the other hand, for formulas (25)–(28), we further have

At the same time, from the character of Itô integrals, we can obtain that

By substituting (22)-(23) into (20) and considering (36), then taking expectation on both sides of (20), and then using (38), we can getBy Lemma 8, there exist nonnegative functions λ 1(t) and λ 2(t) satisfying λ 1(t) + λ 2(t) = 1 such thatSubstituting (43) into (42), then (42) can be rewritten aswhere θ 1(t) = τ(t)/τ ≥ 0,  θ 2(t) = (τ − τ(t))/τ ≥ 0.

So we can get that the following matrix inequalities hold: By virtue of Lemma 7, (45) and (46) are equivalent to (13) and (14), respectively, so we can get thatthen integrating on both sides of (47) from 0 to t, we can obtainIt indicates that system (1) is stochastically passive in the sense of Definition 4. This completes the proof.

Remark 11 .

In [42], the delay interval is divided into three subintervals, which are [−τ 2, −τ(t)], [−τ(t), −τ 1], and [−τ 1, 0]. In this paper, the new Lyapunov function proposed in Theorem 10 is based on the decomposition of delay interval [−τ, 0] into four subintervals, which are [−ατ(t), 0], [−τ(t), −ατ(t)], [−φ(t), −τ(t)], and [−τ, −τ(t)]. By using the lower bound and upper bound of delay derivative , the idea of delay fraction can be successfully applied to cases of both constant and time-varying delay, so less conservatism results can be expected.

When the leakage delay is constant, namely, δ(t) = δ and ρ = 0, neural network system (1) will become the following model:This system has been studied in [42]; then for system (49) we have the following Corollary 12.

Corollary 12 .

Given scalars τ > 0, 0 < α < 1, 0 < β < 1, λ > 0, and 0 < μ and proper matrices Σ  (i = 1,2, 3), the SNNs described by (49) are stochastically passive in the sense of Definition 4, if there exist positive matrices P > 0, Q > 0  (j = 1,…, 5), and R > 0  (l = 1,…, 5), positive diagonal matrices F > 0(j = 1,2), positive constants γ > 0, ϵ 1, ϵ 2, and real matrices , and of appropriate dimensions such that the following LMIs hold: where and are defined as replacing in by μ and −μ, respectively; the other terms have the same expression as that in Theorem 10.

It is well known that the Markovian jump systems (MJSs) are a special class of hybrid systems, which have the advantage in modeling the dynamic systems subject to abrupt variation in their structures, such as component failures and sudden environmental disturbance. Many researches about the stability analysis, impulsive response, and state estimation on the neural networks with Markovian jumping parameters have been obtained; see [44-47] and references therein. Recently [48] has studied the passivity of stochastic neural networks with Markovian jumping parameters; the same method can be used to a system with Markovian jumping parameters and it still leaves much room to reduce the conservatism, which motivates our aim.

Let r , t ≥ 0, be a right-continuous Markov chain defined on a complete probability space (Ω, ℱ, P) and taking discrete values in a finite state space S = {1,2,…, N} with generator Π = (π ) given bywhere Δ > 0 and π ≥ 0 is the transition rate from i to j while π = −∑ π .

For the purpose of simplicity, in the sequel, for each r = i ∈ S, A(r ), W 0(r ), and W 1(r ) are denoted by A , W 0, W 1, and so on. Throughout the paper, we assume that ω(t) and r(t) are independent. Then when the leakage delay does not exist, system (1) will become the one as follows:This system has been studied in [43] and good results have been obtained. In order to testify the effectiveness of our methods, we give the following Theorem 13.

Theorem 13 .

Given scalars τ > 0, 0 < α < 1, 0 < β < 1, λ > 0, and 0 < μ and proper matrices Σ  (n = 1,2, 3), the SNNs described by (1) are stochastically passive in the sense of Definition 4, if there exist positive matrices P > 0, Q > 0, Q > 0  (j = 1,…, 5), and R > 0  (l = 1,2,…, 5), positive diagonal matrices F > 0(j = 1,2), positive constants ϵ 1, ϵ 2, γ > 0, and real matrices , and of appropriate dimensions such that the following LMIs hold: where and are defined as replacing in by μ and −μ, respectively. One has

Choose a Lyapunov-Krasovskii functional candidate as V(x(t), t, i) = ∑ 5 V (x(t), t, i), wherewhere φ(t) = τ(t) + β(τ − τ(t)) and 0 < α < 1 and 0 < β < 1.

By the same method as that in Theorem 10, we can get that the following inequalities hold:wherewhereSo by virtue of Lemma 7 and the same proof method of Theorem 10, we can get that system (55) is stochastic passive.

4. Numerical Example and Simulation

In this section, three numerical examples are presented to demonstrate the effectiveness of the developed method on the obtained passive results.

Example 1 .

Consider neutral stochastic neural networks (1) with the following parameters (Example  5 in [42]):

Take the activation function as f 1(x(t)) = f 2(x(t)) = tanh⁡(x), so it can be verified from Assumption 1 that F 1 = diag⁡{0,0} and F 2 = diag⁡{−0.5, −0.5}, and by using of the Matlab LMI Control Toolbox, we find out a solution to LMIs (12), (13), and (14) as follows:In order to testify the effectiveness of our proposed method, many experiments have been done and the upper bounds of delays δ and τ are listed from Tables 1 to 3, where “–” means that LMIs (12)–(14) has no feasible solution. Table 1 shows the maximum allowable upper bound δ for different values of ρ , which means that the bound of the derivative of the leakage-time-varying is very effective and plays an important role in obtaining the feasible results.

Table 1

Maximum allowable bounds of δ for different ρ values and α = 0.6, β = 0.6, τ = 1.0, and μ = 0.6.

ρ σ	0.1	0.4	0.45
Theorem 10	0.5176	0.3251	—

Table 3

Allowable upper bounds of τ for different values of δ; ρ = 0.1 and μ = 0.5.

δ	0.1	0.05	0.1	0.4	0.45
τ	0.8367	0.7537	0.6415	0.2593	—

From Table 2, we can see that when μ > 1, the feasible solution can be obtained. From Table 3 we can see that when fixing the value of μ and ρ , the allowable upper value of τ is effected by δ, especially when δ = 0.45; the feasible solution cannot be obtained. Especially, when leakage δ(t) = δ, namely, leakage delay, is constant, the studied system will become system (40), which has been researched in [42]; then we have the following Example 2.

Table 2

Maximum allowable bounds of τ for different μ values of ρ = 0.2 and δ = 0.2.

μ	0	0.5	0.8	1.1
τ	0.8267	0.7342	0.4682	0.3848

Example 2 .

Consider that stochastic neural networks (49) have the same parameters as that in [42], so from Corollary 12, we can have the following research results listed in Tables 4 and 5, which show the effect on τ for different μ and mutual effect between τ and δ.

Table 4

Allowable upper bounds of τ for different values of δ; μ = 0.5.

δ	0.01	0.05	0.1	0.4	0.45
Corollary 12	0.9293	0.7882	0.7023	0.3685	—
[42]	0.0005	—	—	—	—

Table 5

Allowable upper bounds of τ with different values of μ; α = 0.4, β = 0.6, and δ = 0.2.

μ	0	0.2	0.5	0.8	0.9	1.1
τ	0.3025	0.3004	0.2946	0.2886	0.2886	0.2886

From Table 4 we also can see that when the same parameters in (2) and (3) of [42] are taken into account and when λ is set by 1 and δ = 0.1 and R 1 = R 2 = 0.1∗I, by solving (2) and (3) of [42], we can get that maximum value τ is 0.0005, so our method has obtained the less conservatism than that of [42].

In this example, when , Figure 1 shows the state curve with u(t). From Figure 1 we can see that when stochastic disturbance and input exist, the systems with leakage delay are unstable.

Figure 1

State curves of system (1) with input u(t).

Remark 14 .

In [42], the sufficient conditions of passivity about stochastic neural networks are given by LMIs, but the solution is not given out, and the simulation about both stochastic and leakage delay is not discussed, either. In our discussion, the impact of leakage delay on stability of systems is considered.

At the same time, when leakage delay is set by different values, by taking the initial state [2, −1] and using the Matlab software, a state curve is obtained as in Figure 2; Figure 2 shows the state curves of system (49) without input and δ is 0.6 and τ = 0.8; Figure 3 shows the state curves of system (49) without input and δ is 0.2 and τ = 0.8.

Figure 2

State curves of system (1) without input and δ is 0.6.

Figure 3

State curves of system (49) with leakage delay 0.2.

When stochastic disturbance does not exit in system (49), the state simulation curves of (49) are shown in Figure 4.

Figure 4

State curves of system (1) without input.

Figure 4 shows the state curves of system (49) without stochastic disturbance and different leakage delay; from Figure 4, we can see that when the leakage delay exits in the neural networks system, state curve of system oscillates sharply from the start point and then becomes asymptotically stable; at the same time, we can find that the bigger the leakage delay, the more serious the oscillation.

Remark 15 .

In Corollary 2 of [42], the maximum value of time delay is 0.2, when leakage delay is set by 0.1. In our method, combing the simulation curve with the value of δ, when δ is 0.1, the maximum value of time delay which is guaranteeing the fact that system (1) is stable can reach 1.2.

Remark 16 .

In [42], an example has been given to show the effectiveness of passivity criteria, but how the leakage delay affects the stability is not discussed. In our example, simulations have been given and proved that leakage delay can cause effect on the stability of neural networks.

Example 3 .

Consider a two-neuron stochastic neural network with Markovian jump parameters and mixed time delays (56) with the following parameters [43]:Let the Markov process governing the mode switching have generatorWe take f (x ) = tanh⁡(x ), α = 0.5, μ = 0.5, β = 0.6, and τ(t) = 0.7 + 0.1cos⁡(t); by solving the LMI in Theorem 13, the following feasible solutions can be obtained:At the same time, in order to testify the less conservatism of our method, the allowable upper bounds of τ with different values of μ have been compared with that in [43]; the results are shown in Table 6.

Table 6

Allowable upper bounds of τ with different values of μ; α = 0.5 and β = 0.6.

μ	0	0.4	0.8	1.1
[43]	0.3025	0.2946	0.1886	0.1826
Theorem 13	0.8226	0.6357	0.4786	0.2839

Mode 1

Mode 2

From Table 6, we can see that even μ > 1; our methods can improve existing research results.

On the other hand, we select x(0) = [0.6, −0.4] and u(t) = [sin⁡(t), t∗cos⁡(t)], and the following simulation results can be obtained. Figure 5 shows the state curve of system (56) with u(t), Figure 6 shows the state curve of system (56) without u(t), and Figure 7 shows the state switching modes of system (56), so the simulation results further prove that the two-neuron stochastic neural networks with Markovian switching parameters is passive in the sense of Definition 4.

Figure 5

State curves of system (55) with input.

Figure 6

State curves of system (55) without input.

Figure 7

The Markov chain of system (55).

5. Conclusions

In this paper, we have investigated the passivity problem for a class of stochastic neural networks systems (SNNs) with varying delay and leakage delay. By constructing a novel Lyapunov functional and utilizing the delay fractionizing technique, new passivity conditions have been established to achieve the passivity performance. Moreover, in derivation of the passivity criteria, it is assumed that the description of the activation functions is more general than the commonly used Lipschitz conditions; the time-varying delay is divided into several subintervals; two adjustable parameters are introduced so that more information about time delay has been utilised. Finally, examples and simulations are provided to illustrate the impact of leakage delay on stability of neural networks and the less conservatism of the developed approach.