Machine Learning for Modeling the Singular Multi-Pantograph Equations.

In this study, a new approach to basis of intelligent systems and machine learning algorithms is introduced for solving singular multi-pantograph differential equations (SMDEs). For the first time, a type-2 fuzzy logic based approach is formulated to find an approximated solution. The rules of the suggested type-2 fuzzy logic system (T2-FLS) are optimized by the square root cubature Kalman filter (SCKF) such that the proposed fineness function to be minimized. Furthermore, the stability and boundedness of the estimation error is proved by novel approach on basis of Lyapunov theorem. The accuracy and robustness of the suggested algorithm is verified by several statistical examinations. It is shown that the suggested method results in an accurate solution with rapid convergence and a lower computational cost.

1. Introduction

The application of multi-pantograph differential equations (MDEs) is expanding into various branches of science such as modeling of cell-growth [1], electrodynamics [2], number theory [3], electrodynamics, astrophysics [4], atomic physics [5], among many others.

Recently, due to the importance of MDEs, the solving of these equations have been frequently considered in the literature and many numerical and analytical methods have been presented [6]. For example, in References [7,8,9], a homotopy approach and power series are developed for solving linear MDEs and coinciding of the estimated solution with the exact solution is investigated. In Reference [10], the spectral tau method is studied and the convergence of the presented approach is investigated by norm. In Reference [11], by obtaining the fractional integral of Taylor wavelets in the sense of Riemann-Liouville definition, an estimated solution is presented for fractional MDEs. In Reference [12], by the Bessel and block-pulse functions, a numerical solution is suggested and it is shown that the increasing of Bessel functions improves the accuracy. In Reference [13], on the basis of topological degree theorem, an analytical method is developed. In Reference [14], by the use of residual power series, an analytical solution is obtained and the efficiency of using residual power series is compared with Chebyshev and Boubaker polynomials. In Reference [15], Adomian decomposition approach is used to construct a solution algorithm for MDEs with fractional-order in the sense of Caputo definition. In Reference [16], by the use of Riemann Liouville fractional derivative and integral definitions, some operational matrices are constructed and then on basis of Jacobi polynomials, an analytical solution is presented. In Reference [17], the collocation method by the use of Boubaker polynomials is developed to convert the problem into a nonlinear system, and then by solving the reduced nonlinear system, an approximated solution is presented. In Reference [18], the spectral tau approach using Jacobi polynomials is improved to solve MDEs. The shifted Gegenbauer-Gauss collocatio technique is introduced in Reference [19], for functional-differential equations. The exponential Jacobi spectral and Jacobi collocation approaches are developed in References [20,21].

Recently, fuzzy logic systems (FLSs) and machine learning algorithms are widely applied on engineering problems [22,23,24,25,26,27,28]. However, to the best knowledge of the authors, the solution of singular MDEs by FLSs and machine learning algorithms has not been studied in the literature. However, quite rarely, some neural methods have been presented for conventional MDEs. For example, in Reference [29], a neural network (NN) is learned by genetic algorithm to find a solution for a pantograph system. In Reference [30], a simple NN is used to find an approximated solution for ordinary differential equations and its accuracy is compared with the analytical solution. In Reference [31], by the method of Lagaris et al, a neural approach is developed for solving MDEs and its efficiency is proved. Now days, the computation techniques in software engineering are used in various branch of scientific problem such as multimedia systems [32,33], security systems [34], forecasting problems [35], stock market prediction [36], control systems [37], internet of things [38], and so on. However these effective techniques quite rarely are applied on MDEs.

Considering the above motivation, in this paper a new approach using T2-FLSs is presented for the solving of singular MDEs. Unlike the aforementioned NN-based methods [29,30,31], the optimization is done by a low computation cost and stable algorithm. The proposed learning algorithm is on the basis of stable SCKF. For the first time, a new approach on the basis of the Lyapunov theorem is suggested to analyze the convergence and closed-loop stability. By several statistical examinations, such as root mean square error (RMSE), inequality coefficient of Theil index (TIC), variance (VAR), fitness (FIT), interquartile range (IR), median (Med), minimum (Min) and mean of absolute error, the accuracy of the suggested method is shown. The main contributions are:

A new numerical method is proposed for solving singular MDEs.

For the first time, a type-2 fuzzy logic based approach is formulated to find an approximated solution.

A new approach on the basis of the Lyapunov theorem is introduced for convergence and stability analysis.

Square root cubature Kalman filter is developed for the optimization of the suggested solver.

Several statistical examinations are presented to demonstrate the accuracy and stability.

The paper organization is as follows. The problem is formulated in Section 2. The suggested T2-FLS is illustrated in Section 3. The learning algorithm is presented in Section 4. The stability is investigated in Section 5. The evaluation indexes are described in Section 6. The simulation results are provided in Section 7, and finally the main outcomes are summarized in Section 8.

2. Problem Formulation

The suggested solver is designed on the basis of fuzzy systems and SCKF. The general diagram of the suggested solution approach is shown in Figure 1. The problem is described as: where the initial conditions are and . and are nonlinear functions. If there is a singularity in and , then both sides of (1) are multiplied by and . The parameters of T2-FLS should be learned such that the estimated solution is to be converged to the exact solution . Then the estimated satisfies: The cost function is defined as follows: where and N is the number of samples.

Figure 1

Block diagram of the proposed solver.

3. T2-FLS Structure

The structure of T2-FLS is shown in Figure 2. The details are given as follows:

Figure 2

The structure of suggested T2-FLS.

Get the input t.

The input t is mapped into time range .

The is divided into M section and for each section a Gaussian membership function (MF) with mean and variance is considered.

The upper and lower firing rules are computed as:

The normalized rule firings (type-reduction by the Nie-Tan approach [39]) are obtained as:

The output is obtained as: where M is number of rules, is the vector of rule parameters. From (7), is computed as:

From (8), is:

Equation (9), can be rewritten as:

Then from (8) and (10), is rewritten as:

Similarly, from (11), is obtained as:

Considering (10), the Equation (12) is rewritten as:

4. Learning Method

The suggested T2-FLS is optimized through the SCKF. To apply SCKF on learning of T2-FLS such that the cost function (3) to be minimized, the following state-space representation is taken to account: where and are the Gaussian noise with covariance R and Q and zeros mean and is the vector of parameters of T2-FLS that includes rule parameters: The learning algorithm is presented as follows.

Consider error covariance as at sample time and compute cubature points , as: where, M is the number of rules and is:

For each in (16), evaluate the cost function J as: where, .

From (18), estimate as the mean of :

Define as:

From (20), compute the square-root of covariance matrix as: where represents triangularization and is the square root of .

Compute cross-covariance as: where

Obtain Kalman gain as:

Update as:

Update error covariance as:

5. Stability and Convergence Analysis

To prove the stability and convergence of the suggested algorithm, the Lyapunov approach [40,41] is used. To apply Lyapunov approach, the following Lyapunov function is defined: Time difference of V, results in: Considering small sample time, can be simplified as: From (13), is obtained as: From (11), is written as: From (7), is computed as: From (29) and Equations (31) and (32), one has: From (25) and (33), one has: From the fact that , it is concluded that and from the Lyapunov theorem, the stability and boundedness of the cost function is derived.

6. Evaluation Index

To evaluate the accuracy and robustness of the suggested algorithm, the following indexes are defined. where N is the number of sample times, and are the exact and estimated solutions and , and are root mean square error, inequality coefficient of Theil index, and variance, respectively.

7. Simulations

By several statistical analyses, the accuracy of the suggested algorithm is examined.

For the first examination, an SMDE is considered as: where where, The time range

For the accuracy of the suggested approach to be well seen, the values of interquartile range (IR), median (Med), minimum (Min) and mean of absolute error at each sample time are provided in Table 1. One can see that the values of mean and IR items are in range of to that indicate an accurate and robust solution.

Table 1

Example 1: Statistical analysis.

t	Min	Mean	Med	IR
0	0.0218	0.0776	0.0823	0.0451
0.0500	0.0228	0.0786	0.0830	0.0448
0.1000	0.0223	0.0788	0.0835	0.0455
0.1500	0.0200	0.0774	0.0827	0.0473
0.2000	0.0154	0.0736	0.0799	0.0498
0.2500	0.0089	0.0672	0.0746	0.0519
0.3000	0.0014	0.0589	0.0675	0.0523
0.3500	0.0040	0.0505	0.0595	0.0520
0.4000	0.0001	0.0430	0.0514	0.0500
0.4500	0.0030	0.0373	0.0434	0.0416
0.5000	0.0039	0.0323	0.0354	0.0328
0.5500	0.0003	0.0279	0.0278	0.0202
0.6000	0.0006	0.0242	0.0231	0.0139
0.6500	0.0042	0.0211	0.0209	0.0129
0.7000	0.0026	0.0179	0.0167	0.0159
0.7500	0.0001	0.0149	0.0124	0.0138
0.8000	0.0019	0.0121	0.0120	0.0127
0.8500	0.0012	0.0092	0.0091	0.0112
0.9000	0.0000	0.0062	0.0057	0.0087
0.9500	0.0003	0.0033	0.0030	0.0053
1.0000	0.0000	0.0000	0.0000	0.0000

In this Example, the following SMDE is considered: where The cost function is: Similar to Example 1, we have 21 rules. The trajectories of the output of T2-FLS (approximated solution), the mean of approximated solutions and exact solution are depicted in

Similar to Example 1, in order for the accuracy of the suggested approach to be well seen, the values of interquartile range (IR), median (Med), minimum (Min) and mean of absolute error at each sample time are provided in Table 2. One can see that the values of mean and IR items are in the range of to that indicate an accurate and robust solution.

Table 2

Example 2: Statistical analysis.

t	Min	Mean	Med	IR
0	0.0004	0.0006	0.0006	0.0001
0.0500	0.0006	0.0009	0.0009	0.0002
0.1000	0.0010	0.0014	0.0015	0.0003
0.1500	0.0013	0.0016	0.0016	0.0002
0.2000	0.0000	0.0008	0.0008	0.0005
0.2500	0.0002	0.0014	0.0013	0.0014
0.3000	0.0013	0.0048	0.0047	0.0025
0.3500	0.0038	0.0090	0.0092	0.0038
0.4000	0.0065	0.0133	0.0137	0.0050
0.4500	0.0090	0.0171	0.0176	0.0059
0.5000	0.0112	0.0204	0.0210	0.0067
0.5500	0.0134	0.0234	0.0240	0.0073
0.6000	0.0155	0.0261	0.0268	0.0077
0.6500	0.0175	0.0285	0.0292	0.0079
0.7000	0.0191	0.0301	0.0309	0.0079
0.7500	0.0201	0.0307	0.0316	0.0075
0.8000	0.0206	0.0306	0.0314	0.0069
0.8500	0.0207	0.0298	0.0306	0.0064
0.9000	0.0205	0.0283	0.0291	0.0057
0.9500	0.0193	0.0255	0.0262	0.0041
1.0000	0.0157	0.0201	0.0205	0.0028

The main hyperparameters of the suggested algorithm are the number of rules and initial covariance matrices. For stability considerations, the initial covariance matrices are chosen to be relatively small. The number of rules is equal to the number of membership functions for input t. To determine the number of rules, the time range

In this section, the performance of the suggested algorithm is compared with other similar techniques in the literature [ The exact solution of (

8. Conclusions

In this paper, a new approach on the basis of fuzzy neural networks and SCKF is introduced for finding a numerical solution for multi-pantograph singular differential equations. The proposed learning method is stable and this property is shown by a new approach on the basis of the Lypunov theorem. Two simulations are provided to demonstrate the efficiency of the designed solver. Several statistical analyses are given to verify the effectiveness of the introduced algorithm such as the analysis of RMSE, Interquartile Range, Theil’s Inequality Index and Variance metrics. The metrics of TIC, RMSE, VAR and FIT are shown in the favorable level and the trajectory approximated solution well tracks the exact solution. Also, the performance of the suggested method is compared with the other similar techniques in the literature. It is shown that the proposed technique results in better accuracy despite less computational cost in contrast to the evolutionary based learning genetic algorithm.

Table 3

Example 3: Comparison.

t	Exact Solution	Proposed Method	Method of Reference [29]
0	1.0000	1.0000	1.0000
0.1	1.1052	1.1051	1.1051
0.2	1.2214	1.2213	1.2213
0.3	1.3499	1.3498	1.3497
0.4	1.4918	1.4917	1.4917
0.5	1.6487	1.6486	1.6486
0.6	1.8221	1.8221	1.8220
0.7	2.0138	2.0137	2.0136
0.8	2.2255	2.2254	2.2253
0.9	2.4596	2.4595	2.4594
1	2.7183	2.7181	2.7181