Artificial neural networks: a practical review of applications involving fractional calculus.

In this work, a bibliographic analysis on artificial neural networks (ANNs) using fractional calculus (FC) theory has been developed to summarize the main features and applications of the ANNs. ANN is a mathematical modeling tool used in several sciences and engineering fields. FC has been mainly applied on ANNs with three different objectives, such as systems stabilization, systems synchronization, and parameters training, using optimization algorithms. FC and some control strategies have been satisfactorily employed to attain the synchronization and stabilization of ANNs. To show this fact, in this manuscript are summarized, the architecture of the systems, the control strategies, and the fractional derivatives used in each research work, also, the achieved goals are presented. Regarding the parameters training using optimization algorithms issue, in this manuscript, the systems types, the fractional derivatives involved, and the optimization algorithm employed to train the ANN parameters are also presented. In most of the works found in the literature where ANNs and FC are involved, the authors focused on controlling the systems using synchronization and stabilization. Furthermore, recent applications of ANNs with FC in several fields such as medicine, cryptographic, image processing, robotic are reviewed in detail in this manuscript. Works with applications, such as chaos analysis, functions approximation, heat transfer process, periodicity, and dissipativity, also were included. Almost to the end of the paper, several future research topics arising on ANNs involved with FC are recommended to the researchers community. From the bibliographic review, we concluded that the Caputo derivative is the most utilized derivative for solving problems with ANNs because its initial values take the same form as the differential equations of integer-order.

Introduction

Artificial neural networks (ANNs) have emerged as a promising alternative to simulate systems due to their successful applications in several engineering and science fields, such as signal processing, image processing, control systems, associative memory, to name a few. Besides, fractional calculus (FC) is an extension and generalization of the integer-order calculus, which its main characteristic is the memory description. When the ANNs are modeled using fractional differential equations (FDE), they are named fractional artificial neural networks (FANNs). The FDE is used for describing the dynamical behavior of the ANNs neurons. Hence, in the last decade, many authors employed FANN for modeling physics and engineering systems more efficiently and accurately.

In this manuscript, we will use the term “ANN involved with FC” to refer to the FANN. Compared with the ordinary ANNs, ANNs involved with FC have important advantages, such as the description of memory and hereditary properties of several processes; and the system performance is enriched due to one more degree of freedom [1]. Fractional-order systems can process information efficiently, improving the simulations of the integer-order systems, finding more accurate results. Since many of the real-world problems can be generally identified and described by the fractional-order models [2], it can be expected the same or better results could be reached using the FANN.

The key aspects of this review are the study and comprehension of the main derivatives related to FC. Several definitions of fractional derivatives have been proposed, for example, Grünwald–Letnikov, Riemann–Liouville, Caputo, Caputo–Fabrizio, and Atangana–Baleanu fractional derivatives. The Riemann–Liouville and Caputo derivatives use the power-law kernel. The Caputo–Fabrizio derivative uses an exponential kernel, and the Atangana–Baleanu derivative utilizes the generalized Mittag–Leffler function as the nonsingular and nonlocal kernel. The equations described by fractional derivatives are highly complex, and there exist lots of analytical and numerical methods to solve them. These techniques have allowed establishing a comparison point between the exact solution and the approximation carried out by the ANN involved with FC. Several numerical or analytical methods that have been developed to solve FDE are, for instance: the Adams–Bashfort–Moulton method, homotopy perturbation method, variational iteration method, Adomian decomposition method, Laplace transform method, among others [3-10].

The uncertainty of parameters affects the modeling and controlling of the systems we are dealing with. Therefore, parameter estimation of ANN involved with FC is crucial for the theoretical study and practical applications [11]. In training ANN involved with FC, the synaptic connection weights between different neurons are adjusted. The weights training is carried out using optimization algorithms. An optimization algorithm is an efficient searching method to solve constrained optimization problems [12]. The various optimization algorithms to train ANN involved with FC include the algorithms based on back-propagation (BP), such as gradient descent algorithm (GD) or Levenberg–Marquardt algorithm (LM) among others, and algorithms based on heuristic methods, such as genetic algorithm (GA), simulating annealing algorithm (SA), particle swarm optimization algorithm (PSO), and so on. At present, synchronization of chaotic fractional-order differential systems becomes a challenging and interesting problem due to its potential applications and the ability to model systems accurately. Amongst all kinds of fractional-order chaos synchronization, the most commonly employed to synchronize ANN involved with FC are projective synchronization, global synchronization, finite-time synchronization, quasi synchronization, and adaptive synchronization. Nowadays, many authors have published works where sufficient conditions are derived, to achieve these types of synchronizations on ANN involved with FC [13, 14]. Many control techniques have been used to show their synchronization, among them: feedback control, adaptive control, sliding mode control, impulsive control, and so on [15]. Moreover, the stability analysis of ANN is one of the most important and active areas of research. Consequently, some improved stabilization methods for different types of nonlinear systems are worthy of further investigation [16]. Stability analysis of several systems has been investigated since they have been successfully applied in some engineering fields, such as signal processing, pattern classification, control, and optimization [14]. In recent years, FC is introduced for the stability analysis of nonlinear systems, allowing us to study the most important stability types, such as exponential stability, finite-time stability, uniform stability, global stability, etc. Several control techniques have been widely used to guarantee stability on ANN involved with FC, among them, sliding mode control, feedback control, and impulsive control [13, 14].

Besides, the ANN involved with FC has been employed in approximation, estimation, control of chaos. Moreover, it has been found an ANN involved with FC with applications in cryptographic, medicine, sustainable energy, images, circuit realization, unmanned aerial vehicles, and robotics.

In the present work, a state of the art review related to the ANN involved with FC is carried out. This paper is organized as follows: Sect. 2 presents a synthesis about the FC applied to ANN; in Sect. 3, the analytical and numerical methods employed to solve the Differential Equations (DE) and FDE that model the concerned systems are reviewed. Subsequently, Sect. 4 presents a thorough overview of the optimization algorithms employed for the training of ANN involved with FC; in Sect. 5, the control strategies employed to synchronize and stabilize ANN involved with FC are summarized. Section 6 shows other important applications of ANN involved with FC. In Sect. 7, some future directions about ANN involved with FC are given. Finally, Sect. 8 presents a summary of the most relevant information of this research.

Mathematical preliminaries

In this section, we present some fractional-order derivatives widely used in the fractional calculus.

Definition 1

Let and . The fractional operator in the Riemann–Liouville sense is given as follows [17]where a and t are the limits of operation of and is the Euler Gamma function.

Definition 2

Let . The fractional operator of Grünwald–Letnikov is given by [17]where,and is the generalized binomial coefficient.

Definition 3

The Liouville–Caputo fractional derivative, which will be named as Caputo derivative in the rest of this paper, is defined as the convolution of the local derivative of a given function with a power-law kernel. Therefore, the derivative of order is defined as follows [17]

Definition 4

Let then, the Atangana–Baleanu fractional derivative in Liouville–Caputo sense is given as [18]where is a normalization function.

There are different analytical and numerical methods to solve DE and FDE. In this section are introduced the methods used to solve the DE and FDE on ANN involved with FC. Also, in this section, a brief discussion is carried out to summarize some of these methods. In [19], a chaotic memristive Hopfield FANN was designed using the 4th order Runge–Kutta numerical method involving the Grünwald–Letnikov derivative. The dynamical properties of the system were studied. And an adaptive sliding mode control was used for the system synchronization. In Ref. [20], the problems on the stability and synchronization of quaternion-valued FANN was investigated involving Caputo derivative, and the Adams–Bashforth–Moulton predictor–corrector method was used to solve the FDE and demonstrate the validity of the theoretical results. On the other hand, in [21], an orthogonal FANN was employed to solve various types of Lane–Emden equations that arise in several physical phenomena. The fractional-order Lane–Emden equation was generalized by considering its derivative in Caputo sense, the analytical approximation to the solution of the FDE was carried out using Adomian decomposition method. In Ref. [22], an orthogonal Jacobi FANN was employed to perform the numerical simulations of nonlinear fractional dynamics based on various types of FDE, the obtained results were compared with other numerical methods, such as the spectral collocation method, meshless method, and reproducing kernel method, to demonstrate the feasibility of the proposed approach.

The stability of impulsive complex-valued BAM FANN with time-varying delays was studied in Ref. [23]. In this work, the Laplace transform of the Mittag–Leffler function was obtained, and the Mittag–Leffler stability of the Caputo fractional derivative was proved.

For homogeneity reasons, each table presented in this paper shows the ANN architectures that involve: time delay, time varying-delays, multiple delays, mixed delays, and leakage delay, followed by the architecture of the ANN.

Table 1 shows the overview of methods used to solve DE and FDE applying ANN involved with FC. The types of the differential equation, the referenced work, the method to solve the DE or FDE, the ANNs architectures, as well as the fractional derivative used, are shown in Table 1, as follows:

Table 1

Methods to solve fractional differential equations

Differential equation	References	Method of solution	ANN architecture	Fractional derivative
DE	[24]	Adams–Bashforth–Moulton method	Feed-forward ANN
	[21]	Collocation method	Orthogonal FANN	Caputo derivative
	[21]	Reproducing kernel method	Orthogonal FANN
	[25]	Runge–Kutta method	ANN
DE	[19]	Runge–Kutta method	Memristive Hopfield FANN	Grünwald–Letnikov derivative
FDE	[26]		ANN
	[11, 27–30]		FANN
	[31]		Cellular FANN
	[32]		Complex-valued FANN
	[33, 34]	Adams–Bashforth–Moulton Method	Dynamic FANN	Caputo derivative
	[35–39]		Hopfield FANN
	[40]		Memristive FANN
	[20]		Quaternion-valued FANN
	[41]		Recurrent FANN
FDE	[42]	Adams–Bashforth–Moulton method	Feed-forward ANN	Riemann–Liouville derivative
	[43]		Nonidentical FANN
FDE	[44]	Adams–Bashforth–Moulton method	Feed-forward ANN	Riemann–Liouville and Caputo derivative
FDE	[45]	Adams–Bashforth–Moulton method	Recurrent FANN	Grünwald–Letnikov derivative
FDE	[3]	Adams–Bashforth–Moulton method	ANN	Atangana–Baleanu in Caputo sense derivative
FDE	[21]	Adomian decomposition method	Orthogonal FANN	Caputo derivative
FDE	[46]	Adomian decomposition method	Feed-forward ANN	Riemann–Liouville and Caputo derivative
FDE	[46]	Chebyshev wavelet method	Feed-forward ANN	Riemann- Liouville and Caputo derivative
FDE	[47–49]	Collocation method	Feed-forward ANN	Caputo derivative
	[22]		Orthogonal Jacobi FANN
FDE	[46]	Differential transform	Feed-forward ANN	Riemann–Liouville and Caputo derivative
FDE	[26]	Euler method	ANN	Caputo derivative
FDE	[3]	Euler method	ANN	Atangana–Baleanu in Caputo sense derivative
FDE	[50]	Finite difference method	Deep FANN	Caputo derivative
	[22]		Jacobi orthogonal FANN
FDE	[51]	Finite difference method	ANN	Riemann–Liouville and Caputo derivatives
FDE	[46]	Homotopy perturbation method	Feed-forward ANN	Riemann–Liouville and Caputo derivative
	[44]		Feed-forward ANN
	[25]		ANN
	[52]		Deep convolutional ANN
	[28, 29, 53–58]		FANN
	[59]		BAM FANN
	[60]		Cohen–Grossberg FANN
	[1, 61]		Complex-valued FANN
FDE	[62]	Laplace transform method	Fuzzy FANN	Caputo derivative
	[38]		Hopfield FANN
	[16, 63]		Memristive FANN
	[15]		Nonidentical FANN
	[64]		Quaternion-valued FANN
	[65]		Quaternion-valued memristive FANN
	[41, 66]		Recurrent FANN
FDE	[67]	Laplace transform method	Backpropagation ANN	Riemann–Liouville derivative
FDE	[68]	Laplace transform method	ANN	Grünwald–Letnikov
FDE	[26]	Legendre method	Feed-forward ANN	Caputo derivative
	[69]		FANN
FDE	[3]	Legendre Method	ANN	Atangana–Baleanu in Caputo sense derivative
FDE	[47]	Shifted Legendre method	FANN	Caputo derivative
	[22]		Orthogonal Jacobi FANN
FDE	[27]	Linear interpolation method	Hopfield FANN	Caputo derivative
FDE	[70]	Linear interpolation method	Hopfield FANN	Grünwald–Letnikov derivative
FDE	[22]	Meshless method	Orthogonal Jacobi FANN	Caputo derivative
FDE	[71]	Mellin transform method	FANN	Riemann–Liouville, Caputo and Grünwald–Letnikov
FDE	[21]	Orthogonal polynomials method	Orthogonal FANN	Caputo derivative
FDE	[69]	Power series expansion method	FANN	Caputo derivative
	[47]		FANN
FDE	[72]	Power series expansion	Cellular FANN	Grünwald–Letnikov derivative
FDE	[22]	Reproducing kernel method	Orthogonal Jacobi FANN	Caputo derivative
FDE	[46]	Variational iteration method	Feed-forward ANN	Riemann–Liouville and Caputo derivative
	[12]		FANN
FDE	[44]	Variational iteration method	Feed-forward ANN	Riemann–Liouville and Caputo derivative
	[73]		Delayed cellular ANN
	[74–76]		Delayed FANN
	[77, 78]		Delayed BAM FANN
	[79–81]		Delayed complex-valued FANN
Delayed FDE	[82]	Adams–Bashforth–Moulton method	Delayed Fuzzy Cellular FANN	Caputo derivative
	[83–89]		Delayed Hopfield FANN
	[90–92]		Delayed memristive FANN
	[93]		Delayed memristive quaternion-valued FANN
	[94]		Delayed quaternion-valued FANN
Delayed FDE	[95]	Adams–Bashforth–Moulton method	Delayed FANN	Riemann–Liouville derivative
	[96]		Delayed competitive FANN
Delayed FDE	[97]	Adams–Bashforth–Moulton method	ANN	Atangana–Baleanu in Caputo sense derivative
Delayed FDE	[98]	Chebyshev orthogonal polynomial method	Chebyshev FANN	Atangana–Baleanu in Caputo sense derivative
	[99]		Hopfield FANN
	[74, 100–102]		Delayed FANN
	[103–105]		Delayed BAM FANN
	[106]		Delayed cellular FANN
	[107]		Delayed Cohen–Grossberg FANN
	[81, 108–112]		Delayed complex-valued FANN
	[23]		Delayed complex-valued BAM FANN
	[80, 113]		Delayed complex-valued Hopfield FANN
	[2]		Delayed complex-valued memristive FANN
Delayed FDE	[114]	Laplace transform method	Delayed competitive FANN	Caputo derivative
	[115]		Delayed coupled FANN
	[84, 86, 87]		Delayed Hopfield FANN
	[90, 116–118]		Delayed memristive FANN
	[119]		Delayed memristive BAM FANN
	[120]		Delayed memristive Cohen–Grossberg FANN
	[121]		Delayed quaternion-valued FANN
Delayed FDE	[98]	Laplace transform method	Chebyshev FANN	Atangana–Baleanu in Caputo sense derivative
Delayed FDE	[98]	Shifted Legendre method	Chebyshev FANN	Atangana–Baleanu in Caputo sense derivative

According to the bibliographic analysis developed in this manuscript, we can affirm that the most used method to solve FDE and Delayed FDE is the Laplace transform method, followed by the Adams–Bashforth–Moulton method, both of them mainly used the Caputo derivative.

Brief analysis about numerical methods

The interest in applying and solving fractional differential equations (FDEs) has increased in the last decades. Methods such as Laplace/Sumudu perturbation [62, 67, 68, 98, 114, 122, 123], Adomian decomposition [46, 124, 125], homotopy perturbation [126] or decomposition have been used to reach this objective. However, these methods are faced with the convergency, the stability, ability to handle strong non-linearities [127] even the presence of a persistent memory [128].

For this reason, linear multistep methods are a powerful option to solve fractional differential equations. The Adams–Bashforth method (ABM) is an efficient numerical scheme that converges toward the exact solution. One can see several types of research whose numerical results were reached by ABM [3, 24, 33, 34, 42, 44, 45, 82, 95, 97]. It can lead to non-local, non-singular kernel fractional derivatives [129]. However, ABM requires several floating-point operations [128], and this has limitations because of the Lagrange polynomial. Dealing with these limitations, Atangana and Araz [130] developed a numerical scheme based on Newton polynomial, which seems to be accurate.

Therefore, the proposal of numerical schemes for solving FDEs is still an open field for new accurate proposals with the ability to handle strong non-linearities with the least computational effort.

Optimization algorithms for training artificial neural networks

This section presents the works found in the review of the state of the art where optimization algorithms are employed for the training of ANN involved with FC. First, the works with a fractional approach are described. Afterwards, the proposals under integer-order operators are described.

Optimization algorithms with fractional approach

In literature, there were found six works where the fractional GD algorithm was implemented, four works based on fractional BP algorithm, and one research where a Darwinian particle swarm optimization algorithm of fractional-order was developed, all of them were used for the training of ANN involved with FC. Next, these works are discussed.

Gradient descent algorithm (GD)

In Ref. [131], the author implemented a fractional GD algorithm to derive the fractional back-propagation through time (FBPTT) algorithm for recurrent ANN, based on the Riemann–Liouville derivative; this fractional algorithm was able to solve three estimation problems, namely: nonlinear system identification, classification of pattern and Mackey–Glass chaotic time series prediction, outperforming the conventional back-propagation through time performance.

Other interesting works were proposed in Refs. [132, 133]. They got a fractional GD back-propagation method based on the Caputo derivative for training an ANN [132] and deep BP ANN [133]. They derived the error function monotonicity, the proposed algorithms presented weak convergence, and numerical simulations demonstrated the competitive performance of the presented fractional models. Compared with classical integer-order models, the fractional models showed significant advantages of memory storage and hereditary characteristics. The authors in Ref. [133] carried out a comparison between two different methods to test their performances, the first was an ordinary BP ANN, and the second one was a fractional-order deep BP ANN. The results of the training and testing showed that fractional-order deep BP ANN has a better performance.

Afterward, as the fractional-order gradient could not converge to the real extreme point. In Ref. [134], the authors designed a new fractional-order gradient method based on Caputo derivative for the BP of convolutional ANN. In this work, the parameters within the layers were updated using the fractional gradient method, but propagations between layers used integer gradients to keep the chain rule. In fact, the proposed fractional-order gradient method guaranteed the convergence to a real extreme point, fast convergence, high accuracy, and ability to escape local optimal point in ANN when compared with integer-order ANN.

Also, Chen et al. [135] implemented an adaptive fractional-order BP ANN. This technique uses the population extremal optimization, as well as the fractional-order GD training algorithms. The method was developed to solve handwritten digit recognition problems. Population extremal optimization algorithms were used to optimize the initial connection weight parameters, and the fractional GD was used to update these connection weight parameters.

Finally, a FANN was proposed in Ref. [136] for the identification of three different systems. The FANN was trained by using the GD algorithm and the Grünwald–Letnikov derivative. In this work, the three systems that we identified are two benchmark systems and one experimental system. The benchmark systems are a hairdryer, consisting of a mesh of resistor wires heating the air at the entrance of a pipeline, and a hysteresis model consisting of the Bouc–Wen model used to represent hysteresis effects in mechanical engineering. Besides, the experimental acoustic duct system is based on an acoustic waves pipeline. The results demonstrated that the fractional gradient descent algorithm allowed accurate estimations with a reduced number of parameters, compared with other works found in the literature where the gradient descent algorithm of integer-order was employed.

Backpropagation algorithm (BP)

In Ref. [137], the authors developed a Hopfield FANN in the form of an analog circuit. To carried out this, they used factorial, as well as steepest descent fractional approaches.

The authors in Ref. [138] developed a fractional-order BP ANN for improving the performance of the ordinary first-order BP ANN, which was trained by an improved fractional-order steepest descent method. The proposed approach showed to be capable of finding the global optimal solutions. The BPFANN was compared with a classic first-order BPANN by means of an example function approximation, fractional-order multi-scale global optimization, and two comparative performances with real data, involving Grünwald–Letnikov derivative. The BPFANN was compared with a classic first-order BPANN. To carry out the methods comparison, they were used an approximation function, a fractional-order multi-scale global optimization method, and two real data sets. The FANN was developed involving the Grünwald–Letnikov derivative. The BPFANN was superior to the classic first-order BPANN in terms of finding the global optimal solution.

Additionally, in Ref. [139], the authors implemented a fractional observer ANN for high complex fractional-order nonlinear systems, involving the Caputo derivative, for estimating the state variables of a fractional-order nonlinear chaotic system with the unknown dynamic model. A new fractional error back-propagation learning algorithm was developed to adapt the weights of the ANN. This method could eliminate the effect of uncertainties and unmodeled dynamics of the system, showing a fast convergence. The fractional observer was better than other observers of integer-order in terms of accuracies for fractional systems, distinguishing external disturbances, and modeling uncertainties more efficiently.

Darwinian particle swarm optimization algorithm (FO-PSO)

In Ref. [24], the authors developed a feed-forward ANN optimized by applying a fractional-order Darwinian particle swarm optimization algorithm (FO-DPSO) to calculate better solutions to the nonlinear second-order ordinary differential equations representing the corneal shape model (CSM). The authors used the Grünwald–Letnikov derivative. Adams’s numerical solver was used as the reference solution. PSO-DPSO was compared with the hybridization between the PSO algorithm and the Active set algorithm (PSO-ASA). The result showed that ANN-based FO-DPSO was more accurate in the solutions with fewer residual errors. Performance matrices like MAD, TIC, and ENSE were used to test the efficiency of the proposed approach, demonstrating that the proposed methodology was better in terms of less number of function evaluations, mean time value, ENSE, TIC, MAD. FO-DPSO was an excellent technique for tuning the unknown weights involved in the solution designed with ANNs.

Optimization algorithms with classical approach

From the bibliographic review, there were found seven works where the classical GD has been used as an optimization algorithm for training the ANN involved with FC. In two research works, the classical BP algorithm was proposed. Moreover, in six works, the classical Levenberg–Marquardt algorithm (LM) was implemented for training the ANNs. On the other hand, some other classical optimization algorithms have been employed with this purpose, such as interior point algorithm (IPA), genetic algorithm hybridized with pattern search algorithm (GA-PS), sequential quadratic programming algorithm (SQP), Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS), chaotic differential evolution algorithm (CDE), simulated annealing algorithm (SA), particle swarm optimization algorithm (PSO), (SA) hybrid with particle swarm optimization algorithm (PSO) obtaining the (PSO-SA) algorithm, particle swarm optimization algorithm hybrid with enhanced fruit fly algorithm (PSO-EFF), particle swarm optimization algorithm hybrid with artificial bee colony algorithm (PSO-ABC), and stochastic inertia weight particle swarm optimization (SIWPSO) algorithm.

All the above-mentioned works are going to be explained in detail as follows:

Firstly, in Ref. [67], the authors proposed a fractional PID controller with self-tuning parameters based on BP ANN. The discretization method and the design method of the controller were discussed. The authors used the Riemann–Liouville derivative to develop the fractional controller. The controller parameters were tuned by an ANN, which was optimized by the GD optimization algorithm. The fractional PID controller was more flexible than the ordinary. The fractional controller allowed the authors to adjust better its dynamical properties than the ordinary PID. Moreover, the fractional PID controller with a self-tuning parameter based on a BP ANN kept features of a normal fractional PID controller. It has better flexibility because an ANN was introduced to develop the self-tuning controller.

Subsequently, a set of fractional differential equations of initial value problems constructed from cosine basis functions with adjustable parameters were solved in Ref. [140] using an ANN and the Caputo derivative. Numerical solutions were obtained for a single FDE, as well as for systems of coupled FDE. The numerical solutions were obtained by training the ANN repeatedly by means of the GD algorithm. Numerical results were in good agreement with the exact solutions of the FDE.

Similarly, Jafarian in Ref. [141] employed the Caputo derivative over a bounded domain to approximate series solutions of a class of initial value FDE. The ANN was used to solve a fractional type ordinary DE. The original differential equation was transformed into a minimization problem, which was solved using an accurate ANN model for computing the parameters accurately. The authors achieved this using the GD procedure for training a feed-forward ANN. The proposed algorithm was an efficient tool for finding the unknown series coefficients. The obtained simulation results were compared with the exact solutions, Chebyshev wavelet method, and the Homotopy perturbation method, reported in the literature. Finally, the authors demonstrated that numerical simulations carried out by the ANN trained using the GD algorithm were similar to the solutions reported in the literature.

On the other hand, a Chebyshev functional link ANN was proposed by Kheyrinataj and Nazemi in Ref. [98] to model linear and nonlinear delay fractional optimal control problems involving Atangana–Baleanu derivative. The trial solutions were approximated by applying the Chebyshev functional link ANN, which was trained using the GD algorithm. This work presented the first application of delay fractional optimal control with mixed control-state constraints employing a fractional-order derivative with the nonlocal and nonsingular kernel using the Chebysehv functional link ANN approach.

Besides, Antil in Ref. [50] proposed a deep FANN for the time-discretization of a fractional in time nonlinear ordinary DE, employing Caputo derivative. The fractional ordinary DE was minimized by the learning algorithm, solving several issues, such as network instability, vanishing, exploding gradients, long training times, and inability to approximate non-smooth functions. Keeping track of history in this manner improved the vanishing gradient problem and could potentially strengthen feature propagation. It was numerically illustrated the improvement in the vanishing gradient issue using of the proposed deep FANN allowing a better learning ability. The deep FANN was better capable of passing information across the network layers maintaining the relative gradient magnitude across the layers, compared to the standard deep ANN and standard Residual ANN. The deep FANN improved the vanishing gradient issue due to the memory effect, and it handled much better the nonsmooth data due to the network’s ability to approximate non-smooth functions.

Finally, the authors in Ref. [49] tested the ability of a perceptron ANN to approximate functions to solve fractional infinite-horizon optimal control problems involving Riemann–Liouville and Caputo derivatives. The authors used the GD algorithm for training the ANN. There were no other reported works about solving this problem in the literature. Numerical simulations demonstrated the feasibility and efficiency of using the proposed method for solving optimal control problems.

The BP algorithm was used for training a Master-Slave FANN based on Caputo derivative in Ref. [27]. The master network was composed of two Hopfield networks, meanwhile and the slave network was a BP network, doing the BP the system error. The Master-Slave FANN showed to have the highest asymptotic convergence rate and the smallest system error compared with Master-Slave ANN of integer-order.

Moreover, in Ref. [142] the authors found the numerical solution of FDE by employing the Chebyshev ANN, Riemann–Liouville, and Caputo derivatives. BP algorithm was used to train the feed-forward Chebyshev ANN. The accuracy of the proposed method was shown by comparing the analytical solutions with the numerical results. The obtained results showed a good agreement with analytical solutions. The comparison results showed that the Chebyshev ANN is a capable tool for solving linear and nonlinear problems.

Nouh [143] modeled the fractional polytropic gas spheres, which have several applications in physics, astrophysics, engineering, and so on. Thus, the fractional Lane–Emden differential equations of the fractional polytropic gas spheres phenomena were solved employing ANN-based on back-propagation training algorithm, reaching the training of the ANN with small errors predicting the values of fractional Lane–Emden functions.

Levenberg–Marquardt algorithm (LM)

Firstly, Efe and Member in Ref. [144] developed an analog PID controller using an approach of feed-forward ANN. The implementation of fractional-order operators in the PID controller was discussed for establishing a robust control for applications in unmanned aerial vehicles (UAVs). The ANN was trained to provide the coefficients of a finite impulse response (FIR) of approximation type. Some trajectories were described properly by the FIR controller, and the feed-forward ANN, which was trained by the LM algorithm.

Next, in Ref. [145] the authors developed a new method to detect unilateral hearing loss (left-sided and right-sided); the fractional Fourier transform (FRFT) was employed to detect hearing loss more efficiently and accurately. The classifier was a feed-forward ANN trained by the LM algorithm. Some magnetic resonance images were obtained from studies with real patients. The combination of fractional Fourier transform, the principal component analysis, and the neural network as the classifier, showed accuracies higher than 95% concerning the experimental data [145].

On the other hand, Zúñiga-Aguilar [3, 97] used an ANN to get the approximated solution of fractional differential equations of Atangana–Baleanu type in Caputo sense with delay and without delay, respectively. In both cases, the network’s parameters optimization was carried out using the LM algorithm. The results of both ANNs were compared with the analytical solutions and the numerical simulations obtained through the Adams–Bashforth–Moulton method. Different performance indices were calculated to show the effectiveness of the ANNs. The ANN’s were able to achieve approximate solutions with good precision and fast convergence.

Subsequently, Kadam et al. developed the artificial ANN approximation of fractional derivative operators such as Grünwald–Letnikov and Caputo fractional derivatives. LM algorithm was used for training the ANN, considering the mean squared error between the outputs of derivatives and the approximations for validation. Thus, the approximations were computationally fast when compared with the numerical evaluation of fractional-order derivatives.

In a recent paper [21], the authors showed the design of a single layer orthogonal ANN for approximating the solutions of different types of Lane–Emden equations in the Caputo sense. The fractional-order Legendre functions in Caputo sense were used as the hidden layer activation function, while the LM algorithm was used to train the ANN. The obtained results were compared with some other numerical methods and with the exact solution, showing that the proposed orthogonal ANN was accurate and feasible.

Finally, Hadian Rasanan in Ref. [22] implemented a fractional ANN. The authors used fractional-order Jacobi functions as the activation function of the hidden layer. And the identity function was used as the activation function of the output layer. The goal of this work was to approximate the solution of FDE and partial FDE involving Caputo derivative. LM was the training algorithm employed. Thereby, the proposed ANN had the ability to reach high accuracy with few neurons. The effectiveness of the proposed ANN was validated applying linear and nonlinear fractional dynamics. The numerical results were compared with the results obtained from other ANN and some numerical experiments, demonstrating that the proposed model is accurate, fast, and feasible.

Interior point algorithm (IPA)

Regarding the use of the IPA algorithm for the training of ANN involved with FC, Asif in Ref. [12] found the solution of fractional systems governed by the initial value problems of the Bagley–Torvik equation employing a FANN trained with the IPA algorithm via Caputo and Riemann–Liouville derivatives. The designed method was evaluated on different initial value problems of the equation. A comparison between the proposed method and several available criteria, such as an exact solution; Podlubny numerical techniques; an analytical solver based on variational iteration method; and a reported solution of stochastic solvers based on hybrid approaches, allowed to verify the effectiveness of the designed method. Concluding, this efficient computational technique based on FANN, optimized with IPA, was able to find the solutions of different variants of Bagley–Torvik equations in a more accurate way than other stochastic techniques.

Also, in Ref. [146] authors found the approximate solutions of nonlinear quadratic systems based on Riccati equations of fractional-order by means of FANN trained with IPA algorithm. The obtained results were compared with the exact solutions proving the effectiveness of the proposal. This method matched more closely with the standard solution obtained from Adams–Bashforth–Moulton method than the modified homotopy perturbation method and enhanced homotopy perturbation method. The average time consumed by the IPA for a run was lower that than other stochastic techniques based on the PSO and GA algorithms.

Genetic algorithm and pattern search (GA-PS)

A new method to train ANN involved with FC was developed in [147], where a fractional-order system represented by Bagley–Torvik equation was solved by means of feed-forward ANN. This new method is based on evolutionary computational, and it is called the GA algorithm hybrid with the PS technique. Besides, in this work, the Riemann–Liouville derivative was used. The proposed method was successfully applied to different forms of the equation, and the results were compared with a standard approximate analytic solution, stochastic numerical solvers, and exact solutions. The GA, PS, and Ga hybrid with PS (GA-PS) optimizer algorithms were compared against each other for evaluating the performances of the training algorithms, obtaining the best results with the GA-PS algorithm.

Sequential quadratic programming algorithm (SQP)

In Ref. [44], the authors applied a feed-forward ANN and SQP algorithm for the training of weights to obtain the solution of nonlinear quadratic Riccati FDE involving the Riemann–Liouville and the Caputo derivatives. The obtained results with the proposed methodology coincided with the exact solution based on the Adams–Bashforth–Moulton technique. Even, the results were more accurate than the obtained with both the modified homotopy perturbation and the enhanced Homotopy perturbation methods showing the effectiveness of the proposed scheme.

Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS)

An adaptive fractional-order PID controller using ANN was designed based on auto-tuning neurons, involving Caputo derivative in [51]. The Nelder–Mead simplex search method and BFGS algorithm were used for the parameters tuning. The proposed controller was more robust in comparison with conventional controllers [51].

Also, the BFGS quasi-Newton algorithm was employed for training a perceptron ANN involving Caputo derivative in Ref. [26] and [148] to solve FDE and approximate the solution of a fractional optimal control problem, respectively. In [26], the authors validated their method by solving different types of multi-term FEEs. In Ref. [148], the authors validated their proposal by comparing their results with other investigations found in the literature.

Chaotic differential evolution algorithm (CDE)

In Ref. [149], the authors implemented a wind turbine pitch control for regulating the speed of the rotor and power production. The authors proposed a fractional-order PID combined with a radial basis function ANN to improve the performance and alleviating the mechanical loads. The ANN was trained with the CDE algorithm. The fractional-order PID presented better performance and robustness when comparing with other controllers. Moreover, the FOPID controller alleviated mechanical loads in a better way, compared with other control techniques, such as the PID, PI, radial basis function PI, and fractional-order PI controller.

Simulated annealing algorithm (SA)

The chaotic behavior of fractional-order Chua’s system was studied by means of fractional Laplace transform, the activation function for the ANN was the Mexican hat wavelet. The ANN was trained using SA and the analytical solution for the system could be approximated. The accurate approximated solutions, the phase plots of the Lyapunov exponent spectrum, and bifurcation maps of the dynamical evolution of fractional Chua’s system were achieved. Mexican Hat Wavelet-based ANN with SA and fifth-fourth Runge–Kutta method, were proposed to attain the solutions of fractional Chua’s model. Using Caputo derivative, cubic nonlinear was solved efficiently and the Chua’s circuit variables were optimized for different fractional values, in [25].

Subsequently, in Ref. [150] the authors found the numerical solution of delayed FDE based on the application of neural minimization using Chebyshev simulated annealing ANN and Legendre simulated annealing ANN. Chebyshev and Legendre polynomials were used with SA to reduce mean square error and get more accurate numerical approximations. This work was based on the functional link ANN with optimization through thermal minimization. Caputo’s definition was employed for calculating the fractional derivative in the subsequent procedure, and the learning methodology used in this work was the SA algorithm. The obtained results were validated using various experiments, numerically. And graphically with error analysis to demonstrate the accuracy and efficiency of the proposed approach.

Particle swarm optimization (PSO)

In Ref. [34], the authors presented a fractional-order dynamic ANN trained by the PSO algorithm for identifying the Damavand tokamak plasma behavior using the Caputo derivative. The system stability was demonstrated based on the Lyapunov-like analysis. The performance of the proposed approach was compared with experimental data and the integer-order ANN approach. The comparison results showed that the fractional-order dynamic ANN was higher accurate than the dynamic ANN of integer-order.

Zhang and Yang in Ref. [151], studied the optimal quasi-synchronization problem for delayed memristive FANN involving Caputo derivative. Fractional-order inequalities and aperiodically intermittent controllers were proposed to guarantee the quasi-synchronization of the system. Mittag–Leffler function allowed to get the stability result of the fractional-order delayed system. Synchronization of the delayed memrisitve delayed FANN was ensured thanks to matrix inequalities. The control parameters were optimized, and the smaller control energy was obtained applying the PSO algorithm. Simulation examples showed the correctness of the proposed method.

Aslipour and Yazdizadehin in Ref. [34] optimized a dynamic FANN involving Caputo derivative. The authors used the PSO algorithm to identify the behavior of a wind turbine in operation. The results obtained from dynamic FANN were compared with the results obtained from dynamic ANN of integer-order, highlighting that the fractional method was more accurate.

Particle swarm optimization-simulated annealing algorithm (PSO-SA)

In Ref. [46], the authors developed a feed-forward ANN used for approximating the solution of nonlinear Riccati FDE using Riemann–Liouville and Caputo derivatives. The network training was carried out using the particle swarm optimization algorithm (PSO) hybridized with the simulated annealing algorithm (SA). The results were compared with the standard approximate analytical method, the stochastic numerical solvers, and exact solutions. The weights training was proved with other stochastic algorithms, such as SA, GA, GA hybridized with SA (GA-SA), PSO, and PSO hybridized with SA (PSO-SA). The best optimization results were obtained by PSO-SA algorithm.

In Ref. [42], the PSO-SA algorithm was employed for the training of a feed-forward ANN. In this case, the ANN approximated the mathematical model of FDE using Riemann–Liouville and Grünwald–Letnikov derivatives. Comparison between the obtained results and the available exact solutions, analytic solutions, and standard numerical techniques (including both deterministic and stochastic approaches) was carried out, showing that This approach was properly employed to solve different problems associated with linear and nonlinear ordinary FDE. The training of weights was implemented using PSO, GA, GA-SA, and SA algorithms, but the best results were obtained using the PSO-SA algorithm.

Particle swarm optimization and enhanced fruit fly (PSO-EFF)

In Ref. [68], authors developed a nonlinear neural fractional-order proportional integral derivative controller based on ANN and involving the Grunwald–Letnikov derivative, applied to the motion control of a nonholonomic differential drive of a mobile robot. The hybridization of a modified adaptive PSO and the EFF optimization algorithms were used for tuning the parameters of the fractional-order PID controller based on ANN. The fractional-order PID controller decreased the control signals that drive differential drive mobile robot motors by approximately 45% compared with the PID based on ANN of integer-order, and thus reduced the energy consumption in circular trajectories. Numerical simulations demonstrated that the performance of the designed fractional controller was excellent compared with nonlinear controllers of integer-order on the trajectory tracking of the differential drive mobile robot with different trajectories as study cases.

Particle swarm optimization and artificial bee colony (PSO-ABC)

In the current year, Mohammadzadeh and Kayacan in [152] developed an adaptive fractional-order fuzzy control method in the sense of Caputo’s definition to control the frequency in an AC microgrid (MG). A sequential general type-2 fuzzy system based on the radial basis ANN was presented for online modeling of the frequency response of the MG. The parameters were optimized using of a hybridized approach between PSO and ABC algorithms. The learning algorithm was examined using white noise as the control input. It was demonstrated that the proposed identification scheme results were in good performance even in a noisy environment. The results had robust performance in the presence of variation of solar radiation, wind speed, load disturbance, and time-varying dynamics of the other units of MG. The proposed control approach was compared with the conventional PI controller, classic fuzzy, and PSO-fuzzy PI controllers. The results showed that the proposed scheme has better performance.

Stochastic inertia weight particle swarm optimization (SIWPSO)

Optimization of synchronization for delayed memristive FANN was investigated in Ref. [153], involving Caputo derivative. It was designed an appropriate controller where the drive system was able to synchronize with the response system. Synchronization conditions were obtained thanks to the linear matrix inequality, along with fractional-order Lyapunov methods. The FANN was trained using of the SIWPSO algorithm, the target function was the minimal sum of control energy expressed by the integral square error-index, where the Riemann–Liouville derivative was used to get the approximated value of the target function. It was obtained a better controller with low control energy and integral square error (ISE) index. The optimal control parameters of the proposed model were obtained by computing the SIWPSO algorithm, which was an improved intelligent algorithm. A simulation was provided to demonstrate the feasibility of the proposed results.

Synchronization and stabilization of ANN involved with FC

The FC applied to ANNs allows simulating systems more accurately than by the classical approach, due to the FC properties as the nonlocality and the memory description of FC. From the bibliographic review, we can affirm that the main applications of FC applied to ANN are system stabilization, systems synchronization, and the training of these systems through optimization algorithms.

This section is organized as follows: first, the control strategies carried out to guarantee the synchronization and stabilization of ANN involved with FC are summarized. Second, other examples of stabilization reached using FC on ANN are overviewed.

Control strategies employed to synchronize and stabilize ANN involved with FC

Several authors have used control strategies to synchronize and stabilize different architectures of ANN involved with FC. A brief overview of them will be described as follows:

On the one hand, in [11, 13, 55, 154–161] authors employed adaptive control to reach: projective, mixed projective synchronization, and synchronization using the Caputo derivative.

On the other hand, sliding mode control was employed in [15, 58, 162–166] to reach: projective, global projective, mixed projective synchronization, and synchronization using the Caputo and Riemann–Liouville derivatives.

Also, feedback control was found in several works on ANN involving FC. This control strategy was applied in Ref. [20, 43, 53, 56, 59, 61, 90–92, 96, 116–119, 154, 167–183], and aiding to achieve: hybrid projective, finite-time projective, projective, quasi-projective, quasi-uniform synchronization, stability, finite-time stability, synchronization, and global stability using Caputo and Riemann–Liouville derivatives.

Other control strategies, such as adaptive control, impulsive control, or washout filter control were employed in [101, 184–187] to attain: projective, adaptive, global, chaotic synchronization, and global stability, using the Caputo derivative and Grünwald–Letnikov derivatives.

The most used fractional derivative to guarantee the synchronization of these types of systems is the Caputo derivative, followed by the Riemann–Liouville and the Grünwald–Letnikov derivatives, respectively. We found that the Grünwald–Letnikov derivative was only used in one work [187].

The information given in the above sections is summarized in Tables 2 and 3. The tables show the relationship among the control strategy, ANN architecture, the fractional derivative, and the goal achieved (application), as follows:

Table 2

Control strategies employed to synchronize and stabilize ANN involved with FC

Control strategy	References	ANN architecture	Fractional derivative	Application
Adaptive control	[188]	ANN		Synchronization
	[156]	Chebyshev ANN		Synchronization
	[161]	Fuzzy ANN		Synchronization
	[13, 160]	FANN		Global projective synchronization
	[11, 154]	FANN		Global synchronization
	[55, 155]	FANN		Synchronization
	[189]	Chaotic FANN		Approximation and stability
	[82]	Fuzzy cellular FANN	Caputo derivative	Global stability and synchronization
	[159]	Memristive FANN		Projective synchronization
	[190]	Memristive FANN with leakage delay		Synchronization
	[191]	Radial basis function FANN		Synchronization
	[158]	Delayed FANN		Mixed projective synchronization
	[192]	Delayed BAM FANN		Stability and synchronization
	[157]	Delayed fuzzy FANN		Projective synchronization
	[109]	Delayed complex-valued FANN		Quasi-projective and complete synchronization
	[193]	Delayed competitive FANN		Projective Synchronization
Adaptive sliding mode control	[194]	Hopfield FANN	Caputo derivative	Stability
	[195]	Delayed fuzzy FANN		Projective synchronization
	[52]	Deep convolutional ANN		Robotic manipulators
	[196]	Deep recurrent ANN		Synchronization
	[165]	Radial basis function ANN		Stability
	[197]	Radial basis function ANN		HIV infection model
	[198, 199]	Recurrent ANN		Approximation
	[198]	Recurrent ANN		Stability
Sliding Mode Control	[58]	FANN	Caputo derivative	Synchronization
	[200]	Chaotic FANN		Syncrhonization
	[201]	Memristive MAM FANN		Fixed-time synchronization
	[15]	Nonidentical FANN		Projective synchronization
	[166]	Delayed Hopfield FANN		Adaptive synchronization
	[162]	Delayed nonidentical FANN		Projective synchronization
	[164]	Hopfield FANN		Finite-time stability
Sliding Mode Control	[163]	Nonidentical FANN	Riemann–Liouville Derivative	Global projective synchronization
Sliding Mode Control	[19]	Memristive Hopfield FANN	Grünwald–Letnikov derivative	Synchronization
Pinning Control	[202]	Delayed BAM FANN	Caputo Derivative	Quasi-pinning synchronization and stability
	[107]	Delayed Cohen–Grossberg FANN		Stability and pinning synchronization
Feedback pinning control	[40]	Memristive FANN	Caputo derivative	Quasi-synchronization
	[56, 171]	FANN		Stability and synchronization
	[203]	FANN		Robust finite-time cost control
	[29]	FANN		Hybrid projective synchronization
	[181]	FANN		Global synchronization
	[204]	Delayed FANN		Global synchronization
	[173]	Delayed FANN		Synchronization
	[90]	Delayed FANN		Hybrid projective synchronization
	[154]	Delayed FANN		Synchronization
	[205]	Delayed FANN		Projective synchronization
	[59]	BAM FANN		Stability
	[206]	BAM FANN		Global Stability
	[168]	Delayed BAM FANN		Synchronization
	[78]	Delayed BAM FANN		Global synchronization
	[207]	Delayed BAM FANN		Finite-time stability
	[78]	Delayed BAM FANN		Global synchronization
	[53]	Chaotic FANN		Synchronization
	[172]	Memristive FANN		Synchronization
	[118, 208]	Delayed memristive FANN		Synchronization
	[177]	Delayed memristive FANN		Finite-time projective synchronization
	[209]	Delayed memristive FANN		Projective synchronization
	[117, 174]	Delayed memristive FANN		Stability and synchronization
	[210]	Delayed memristive FANN		Global stability
	[211]	Delayed memristive FANN		Synchronization
Feedback Control	[116]	Delayed memristive FANN	Caputo derivative	Finite-time synchronization
	[92]	Delayed memristive FANN		Quasi-uniform synchronization
	[91]	Delayed memristive FANN		Quasi-synchronization
	[180]	Delayed memristive FANN		Global synchronization
	[212]	Delayed memristive FANN		Asymptotic stability
	[119]	Delayed memristive BAM FANN		Finite-time synchronization
	[182]	Memristive recurrent FANN		Finite-time synchronization
	[93]	Delayed memrisitve quaternion-valued FANN		Synchronization and stability
	[175]	Cohen–Grossberg memristive FANN		Finite-time stability and synchronization
	[213]	Quaternion-valued FANN		Finite-time synchronization
	[214]	Quaternion-valued BAM FANN		Synchronization
	[215]	Quaternion-valued Memristive FANN		Global stability
	[20]	Quaternion-valued FANN		Global synchronization and global stability
	[216]	Delayed quaternion-valued FANN		Global synchronization
	[176]	Fuzzy cellular memristive FANN		Finite-time stability and synchronization
	[217]	Fuzzy FANN		Asymptotic stability
	[218]	Coupled discontinuous FANN		Finite-time synchronization
	[219]	Delayed non-autonomous FANN		Synchronization
	[114]	Delayed competitive FANN		Global asymptotic stability
	[115]	Delayed coupled FANN with		Robust asymptotic synchronization
	[179]	Radial basis function ANN		Projective synchronization
	[61]	Complex-valued recurrent FANN		Quasi-projective synchronization
	[109]	Delayed complex-valued FANN		Quasi-projective and complete synchronization
	[167, 183]	Delayed complex-valued FANN		Synchronization
	[170]	FANN		Synchronization
	[220]	Delayed FANN		Stability and synchronization
	[221]	Delayed FANN		Synchronization
Feedback Control	[96]	Delayed FANN	Riemann–Liouville derivative	Synchronization
	[43]	Nonidentical FANN		Global synchronization
	[178]	Memristive FANN		Projective synchronization
	[222]	Complex-valued memristive FANN		Global asymptotic synchronization
	[223]	Delayed memristive BAM FANN		Global projective synchronization

Table 3

Control strategies employed to synchronize and stabilize ANN involved with FC

Control strategy	References	ANN architecture	Fractional derivative	Application
Adaptive feedback control	[224]	Competitive FANN		Robust synchronization and stability
	[114]	Delayed Competitive FANN		Global asymptotic stability
	[101]	Delayed FANN	Caputo derivative	Projective synchronization
	[225]	Delayed BAM FANN		Global stability
	[104]	Delayed BAM FANN		Synchronization
	[184]	Delayed memristive FANN		Adaptive synchronization
Adaptive feedback control	[226]	Delayed complex-valued FANN	Riemann–Liouville derivative	Global synchronization and global stability
	[60]	Cohen–Grossberg FANN		Exponential stability and synchronization
	[227]	Complex-valued FANN		Stability and synchronization
	[228]	Memristive BAM FANN		Finite-time impulsive synchronization
Impulsive Control	[229]	Memristive discontinuous FANN	Caputo derivative	Exponential stability
	[65]	Quaternion-valued memristive FANN		Finite-time stability
	[185]	Delayed FANN		Global stability and synchronization
	[230]	Delayed FANN		Almost periodicity and stability
	[186]	Delayed cellular FANN		Global synchronization
Adaptive impulsive control	[231]	Quaternion-valued FANN	Caputo derivative	Stability and impulsive synchronization
Washout filter control	[187]	Delayed FANN with	Grünwald–Letnikov derivative	Chaotic synchronization

ANN involved with FC to reach stability

Several authors in literature stabilized ANN involved with FC, thanks to the FDE employed in these systems. In this sense, different types of fractional derivatives have been used to reach systems stabilization. Following, we will present some related works to systems stabilization using ANN involved with FC.

Caputo derivative was employed to reach the exponential stability, in Refs. [32, 171, 232], the uniform stability in [76, 79, 233–242], the finite-time stability in Refs. [75, 85, 103, 175, 243–248], the stability and bifurcation in Refs. [36, 37, 41, 73, 74, 77, 80, 108, 249–251], and the quasi-uniform stability, fractional input stability, global stability, and global periodicity in Refs. [252-254], respectively.

Similarly, the Caputo and Riemann–Liouville derivatives were employed to reach: the global stability in Refs. [16, 28, 38, 59, 62, 86, 95, 117, 174, 180, 185, 255–259], the stability in Refs. [35, 41, 54, 57, 66, 84, 87, 113, 155, 260–268].

Also, the Riemann–Liouville derivative was used to reach synchronization stability in Refs. [252–254, 269, 270].

Concluding, the Caputo derivative is the most used fractional derivative for investigating the stability of the ANN involved with FC, followed by the Riemann–Liouville derivative. Nowadays, Caputo and Riemann–Liouville derivatives are the only two derivatives used to reach stability in these systems.

Table 4 summarize research works related to ANN involved with FC to reach stability.

Table 4

ANN involved with FC to reach stability

Application	References	ANN architecture	Fractional derivative
Exponential stability	[171]	FANN
	[32]	Complex-valued FANN	Caputo derivative
	[232]	Interval projection FANN
	[271]	Delayed BAM FANN
Uniform stability	[241]	FANN
	[237]	FANN with and without delays
	[76, 242]	Delayed FANN
	[240]	Hopfield FANN
	[239]	Memristive FANN
	[272]	Delayed memristive FANN	Caputo derivative
	[273]	Delayed memristive fuzzy BAM FANN
	[238]	Cellular FANN
	[79, 233, 234]	Delayed complex-valued FANN
	[235, 236]	Delayed BAM FANN
	[274]	Delayed BAM FANN
Finite-time Stability	[248]	FANN
	[75, 85, 246, 275]	Delayed FANN
	[247]	Delayed FANN
	[112, 243, 276]	Delayed complex-valued FANN
	[277]	Delayed complex-valued memristive FANN
	[244]	Delayed complex-valued memristive FANN	Caputo derivative
	[175]	Cohen–Grossberg memristive FANN
	[245]	Delayed Cohen–Grossberg FANN
	[89]	Delayed Hopfield FANN
	[207]	Delayed BAM FANN
	[103]	Delayed BAM FANN
Global stability	[28]	FANN
	[59]	BAM FANN
	[278]	Delayed BAM FANN
	[62]	Fuzzy FANN
	[38, 256]	Hopfield FANN
	[86]	Delayed Hopfield FANN	Caputo derivative
	[16]	Memristive FANN
	[117, 174, 180, 279]	Delayed memristive FANN
	[106]	Delayed Cellular FANN
	[185, 258]	Delayed cellular FANN
	[257]	Delayed complex-valued FANN
Global Stability	[259]	Delayed FANN
	[95]	Delayed FANN	Riemann–Liouville derivative
	[280]	Delayed fuzzy BAM FANN
	[255]	Delayed hybrid BAM FANN
Stability and Bifurcation	[41]	Recurrent FANN
	[36, 37]	Hopfield FANN
	[74, 250, 281]	Delayed FANN
	[251]	Complex-valued Hopfield FANN
	[80]	Delayed complex-valued Hopfield FANN	Caputo derivative
	[108]	Delayed complex-valued FANN
	[77, 249, 282]	Delayed BAM FANN
	[73]	Delayed cellular FANN
Stability	[54, 57, 155, 263, 267]	FANN
	[268]	Delayed FANN
	[260]	FANN with time-varying delays
	[99]	Hopfield ANN
	[35, 39, 261]	Hopfield FANN
	[84, 87]	Delayed Hopfield FANN
	[41, 66]	Recurrent FANN
	[262]	Nonautonomous FANN	Caputo derivative
	[113]	Delayed complex-valued Hopfield FANN
	[266]	Delayed complex-valued memristive FANN
	[105]	Delayed BAM FANN
	[94]	Delayed quaternion-valued FANN
	[265]	Delayed cellular FANN
Stability	[264]	Delayed neutral type FANN	Riemann–Liouville derivative
Monostability and multistability	[283]	FANN	Caputo derivative
Asymptotical stability	[284]	Delayed FANN	Caputo derivative
Quasi-uniform stability	[252]	Delayed FANN	Caputo derivative
Fractional input stability	[253]	FANN	Caputo Derivative
Global stability and global periodicity	[254]	Delayed complex-valued FANN	Caputo derivative
	[269]	Delayed non-autonomous FANN
Synchronization stability	[270]	Delayed Complex FANN	Riemann–Liouville Derivative
Stability and passivity	[285]	Memristive FANN	Caputo derivative

From the four tables shown in this manuscript: it’s important to mention that temporal models with recurrent FANN have been developed in works: [41, 45, 79, 102, 182, 198, 199, 263, 266].

Other interesting applications of ANN involved with FC

From the bibliographic analysis, there were found works where ANN was involved with FC to reach goals, such as approximation of functions, description of chaos, estimation, global dissipativity, periodicity, and modeling heat transfer process. Other researches have been applied to the different areas of science and engineering, such as medicine, image encryption, robotic, among others. The most important works related to these applications will be described in detail in this section, as follows.

ANN involved with FC: applications in approximation of functions

In several research works, there were approximated functions with the aid of ANN involved with FC. Some of these works will be explained in detail below:

The fractional quantitative approximation of real-valued functions involving Caputo derivative was carried out on feed-forward ANN in Ref. [286]. These approximations were derived by establishing Jackson type inequalities, converging the fractional approximation results better than the integer-order scheme.

The fractional differential polynomial ANN was the proposed method to approximate a multi-parametric function with polynomials, involving the Caputo derivative. The generalization depended on the Riemann–Liouville differential operator, and the experimental results demonstrated that the approximation to the exact value with the fractional differential polynomial ANN was quicker than the integer-order method [287].

Liu and Fei in Ref. [165], approximated the nonlinear Dual Radial Basis functions ANN and the upper bound of estimated disturbances, improving the system stability and robustness, involving Caputo derivative. The ANNs weights were updated online to approximate the dual Radial Basis functions ANN structures, applied to a control system.

In Ref. [189], an adaptive control based on ANN was used to approximate unknown nonlinear functions using the fractional Lyapunov stability criterion and the backstepping technique, to control an uncertain fractional-order Chua–Hartley chaotic systems. The ANN was employed to approximate unknown system uncertainties and external disturbances. The numerical simulation was given to demonstrate the effectiveness of the proposed approach.

On the other hand, Lu and Wang in Ref. [288] developed the adaptive ANN tracking control using backstepping technology for the fractional-order chaotic permanent magnet synchronous motor with the immeasurable state, parameter uncertainties, and external load disturbance, involving Caputo derivative. The proposed approach employed a Chebyshev ANN and a state observer to approximate the unknown functions and estimate the unmeasurable state. In this work, the simulation results were presented to demonstrate the correctness of the proposed methodology.

In Ref. [289], authors developed an adaptive ANN control based on command filtered backstepping method for fractional-order permanent magnet synchronous motor with parameter uncertainties and unknown time delays, involving Caputo derivative. The unknown parameters, as well as the load disturbance, were approximated by using ANN. The time delays uncertainties were gotten by employing proper Lyapunov functions. Meanwhile, numerical simulations were given to demonstrate the effectiveness of the proposed method. Comparison among the proposed controller, classical backstepping controllers, and radial basis function ANN terminal sliding mode surface controller was carried out, showing the proposed controller to have a better performance.

Finally, fractional power series ANN for solving: delay fractional optimal control problems and fractional optimal control problems with equality and inequality constraints were developed in [47, 69] respectively, involving Caputo derivative, feed-forward ANNs and according to the Pontryagin minimum principle. In both works, the optimization techniques and collocation methods were proposed to determine the approximate solution of the fractional optimal control problems. The obtained results were compared with the exact solutions and analytical solutions, respectively.

ANN involved with FC: presence of chaos

Next, several types of research where ANN involved with FC have chaotic behaviors will be depicted:

In Ref. [170], a numerical simulation algorithm for FDE was presented involving Riemann–Liouville derivative, where the chaotic phenomena and their control were discussed by numerical simulation. Chaos feedback control was developed, allowing to control and synchronize the FANN system.

Moreover, in Ref. [290] obtained the fractional-order model of delayed cellular ANN for describing chaotic behaviors for fractional-order 0.1 interval. Meanwhile, delay time values for which the chaos occurred were defined using the largest Lyapunov exponents. Riemann–Liouville derivative was employed in this work, and the simulation results demonstrated that the time delay where chaos occurred decreased as the fractional-order decreased too.

Moreover, Kaslik and Sivasundaram Seenith in Ref. [37] investigated the stability, multi-stability bifurcations, as well as the chaos of Hopfield FANN involving the Caputo derivative. In this work the critical values of fractional-order where the Hopf bifurcations occurred were identified, and the stability domain of a steady-state was characterized. The simulation results demonstrated that the chaotic behavior appeared when the fractional-order of the system increased.

In Refs. [31, 83], the authors studied the behaviors of the complex dynamics of a cellular FANN and a delayed Hopfield FANN, respectively. To carried out these studies, numerical simulations involving the Caputo derivative were developed. The systems investigated in these works presented dynamic behaviors, such as periodic and chaotic motions. Furthermore, in both works, the existence of chaotic attractors was demonstrated. This was verified using bifurcation diagrams and phase portraits.

On the other hand, the projective synchronization of chaotic memristive FANN with time-varying delay and switching jumps mismatch involving Riemann–Liouville derivative, was studied in [178]; also, in this work, the chaotic behavior of the memristor-based FANN system was showed.

Subsequently, Luo et al. in Ref. [156] developed an adaptive synchronization methodology combining Chebyshev ANN, extended state tracking differentiator, and adaptive backstepping, to reach the synchronization between the drive system and response system of a fractional-order chaotic arch micro-electro-mechanical system with the uncertain item and time delay under distributed electrostatic actuation. Caputo derivative was employed in this work, and the stability of the closed fractional-order arch micro-electromechanical system was guaranteed based on the fractional-order Lyapunov stability criterion. The simulation results demonstrated the effectiveness of the proposed scheme.

Moreover, in Ref. [291] stability, bifurcation, and chaos of a Memristive FANN with discontinuous memductance functions were investigated, employing Caputo derivative and identifying interesting dynamics, such as chaotic motion, tangent bifurcation, and intermittent chaos. The chaotic attractors were demonstrated to exist over a wide range of some specified parameters.

Otherwise, the chaotic Chua’s attractor of fractional-order was studied in Ref. [169]. In Ref. [292] the Lorenz system was studied, the chaotic attractors of Hopfield FANN in [293], memristor-based FANN in Ref. [210], two-dimensional delayed FANN in Ref. [101], time-delayed inertial FANN in Ref. [220], and nonidentical FANN chaotic behaviors were studied in Ref. [163].

Finally, Han in Ref. [200] performed a composite learning sliding mode control approach to attain the synchronization of chaotic FANN with unmatched unknown parameters, employing Caputo definition. A comparison between the proposed composite learning sliding mode control and the common sliding mode control demonstrated that the proposed composite learning sliding mode control establishes an accurate parameter estimation without the permanent excitation condition having better control performance than the sliding mode control scheme.

Other research related to chaos of ANN involved with FC applied to image encryption will be found later in this paper, [82, 186, 294] and [295].

ANN involved with FC: applications in estimation

The main works about estimation involving ANN and FC will be depicted in the following summary:

On-line state estimation of nonlinear dynamic systems was carried out using Differential FANN. The simulation of two coupled tanks was carried out to demonstrate the feasibility of Differential FANN as a nonlinear systems identifier [296]. In [35], the same author solved a parameter estimation problem for demonstrating the Hopfield FAN existence using the Caputo derivative. In this case, the stability of Hopfield FANN was reached applying an energy-like function analysis.

Besides, in Ref. [11] authors studied the parameter estimation problem of unknown system parameters on FANN involving Caputo and Riemann–Liouville derivatives. Synchronization-based identification method of fractional-order was achieved thanks to the combination of adaptive control and parameter update law, demonstrating the correctness of the obtained results through a numerical example.

Moreover, in Refs. [297, 298] were designed a state estimator and non-fragile state estimator for delayed memristive FANN involving the Caputo derivative, respectively. In work [297], the state estimators’ existence was ensured, and a suitable state estimator for memristive FANN was proposed. Accordingly, based on the fractional-order Lyapunov direct method, some new sufficient conditions were given to guarantee the existence of the state estimator. On the other hand, in Ref. [298] by using the Lyapunov technique, the authors were getting sufficient conditions to ensure the global asymptotic stability of the error model.

A radial basis function ANN was used to estimate the bound of disturbances in Ref. [299]. And an adaptive fractional sliding mode controller with a neural estimator for a class of nonlinear systems also was designed.

A robust control law was designed to guarantee the occurrence of the sliding motion, as well as the Hopfield FANN stabilization involving the Riemann–Liouville derivative in Ref. [164]. Moreover, the system’s unknown parameters were estimated, and the sliding surface to origin finite-time stability was achieved based on the fractional-order Lyapunov theory. An example of Hopfield FANN was presented to demonstrate the effectiveness of the proposed scheme.

The non-fragile state estimation issue for memristive BAM FANN with and without time delays was studied in Ref. [300] by applying the fractional-order derivative in Caputo sense. Based on fractional-order Lyapunov functionals and linear matrix inequalities was ensured the asymptotical stability of the error system.

The quasi-estimation was investigated by Li in Ref. [301] employing fuzzy memristive FANN based on the Caputo derivative. The quasi-estimation was studied through a Laplace transformation, and the quasi-synchronization control was attained due to the designed feedback controller.

Besides, in Ref. [104], the authors studied the Mittag–Leffler state estimator and an adaptive synchronization for delayed BAM FANN by applying the Caputo derivative. An adaptive feedback control was designed, and Mittag–Leffler adaptive synchronization was reached using fractional-order inequality techniques.

Another novel approach in this area is the robust state estimation of complex-valued FANN with uncertain parameters and BAM FANN with norm-bounded uncertainties, investigated in Refs. [278, 302], respectively. Hu in Ref. [302], applied the Riemann–Liouville derivative, and Nagamani in [278] used the Caputo derivative. Since both systems presented time delays, a new linear matrix inequality criterion was obtained to reach the asymptotic stability of the systems’ error. In both cases, numerical simulations were performed to confirm the effectiveness of the proposed schemes.

Finally, in Refs. [120, 303], the authors investigated the finite-time projective synchronization of memristive FANN with mixed time-varying delays and uncertain parameters and the finite-time synchronization of memristive Cohen–Grossberg FANN with time-varying delays, respectively. In these works, the Caputo derivative and feedback controllers were applied. Moreover, the settling times were estimated.

ANN involved with FC: applications in dissipativity

Following relevant works related to dissipativity involving FC applied to ANN will be shown:

First, in Ref. [100], the authors investigated the global dissipativity of delayed FANN and discontinuous activation functions employing Caputo derivative. In this research, sufficient conditions were given to ensure the dissipativity of the model solution. The effectiveness of the proposed scheme was demonstrated by numerical examples.

Second, the dissipativity and global asymptotic stability of delayed complex-valued FANN were investigated in Ref. [257]. In this research, the authors used the Caputo derivative. Numerical simulations showed the effectiveness of the proposed scheme.

Finally, Li in Ref. [304] investigated the dissipativity and the exponential synchronization control of Memristive FANN involving reaction-diffusion terms, the Caputo derivative, and a feedback controller. The proposed scheme results presented fewer conservation effects when comparing with other works found in the literature.

ANN involved with FC: applications in periodicity

The overview of the research works focused on the periodicity involving FC on ANN will be depicted below:

First, in Ref. [71], the authors demonstrated that the fractional derivative of a periodic function cannot be a periodic function with the same period involving Caputo, Riemann–Liouville, and Grünwald–Letnikov derivatives; in this paper a FANN was employed to guarantee the non-existence of periodic solutions in fractional-order dynamical systems.

Subsequently, Wu and Zeng in Ref. [62] derived the S-asymptotic -periodicity and global asymptotic -periodicity of fuzzy FANN, involving Caputo derivative. The difference between integer-order neurodynamic systems and fractional-order neurodynamic systems was shown. Several simulations were performed to demonstrate the effectiveness of the proposed method.

On the other hand, the global stability and global asymptotic periodicity for complex-valued FANN with time-varying delays was discussed in Ref. [254], while for non-autonomous FANN with time-varying delays was addressed in Ref. [269] by applying the Caputo derivative. In these works, the solutions converged to the same periodic function. Numerical examples were given to demonstrate the feasibility of the schemes.

Finally, the global asymptotic -periodicity for a non-autonomous FANN involving Caputo derivative was investigated in [262]. The authors demonstrated that FANN has S-asymptotically periodic solutions. Furthermore, all solutions of FANN globally converge to a periodic function.

Heat transfer process

Only one work related to the FC on ANN with the heat transfer process will be depicted as follows:

The heat transfer process was modeled using a discrete FANN involving Grünwald–Letnikov derivative in Ref. [305]. The experiments and the obtained results showed that the proposed FANN modeled the unknown dynamics correctly.

ANN involved with FC: applications in sustainable energy

ANN involved with FC has been applied in the sustainable energy area, specifically in wind turbine applications [30, 34, 149]. These works will be addressed below:

The behavior of a wind turbine in operation was identified using a variable order FANN and a Dynamic FANN involving the Caputo derivative in Refs. [30, 34], respectively. The proposed methods were evaluated and validated by using experimental data obtained from the wind turbine under operation. Moreover, in Ref. [34], results obtained with the dynamic FANN were compared with the obtained with the integer-order dynamic ANN showing that the fractional approach was more accurate.

Also, in Ref. [149] was presented a fractional-order PID control implementation for regulating both the rotor speed and the power production of a wind turbine. The control scheme was combined with a radial basis function ANN, allowing it to reach better performance and robustness than with the integer-order controllers.

ANN involved with FC: applications in medicine

ANN involved with FC has been applied to the medical field and reported in Refs. [24, 145, 197, 306]. Below we will describe these research works.

In Ref. [145], the detection of left-sided and right-sided hearing loss was carried out using the fractional Fourier transform and a feedforward ANN trained by the Levenberg–Marquardt algorithm. Also, in Ref. [306] the authors developed a pathological brain detection system based on multi-layer perceptron ANN to improve the interpretation of magnetic resonance brain images. In this work, the ANN was the classifier that received the fractional Fourier entropy features extracted from the brain images. The adaptive real-coded biogeography-based optimization was the algorithm implemented to train this ANN, and the proposed method was able to interpret the images with an accuracy of 99.53%, improving the results obtained from other pathological brain detection system.

Subsequently, Sharafian in Ref. [197] used a radial basis function ANN with a sliding mode observer for modeling the uncertainties of the human immunodeficiency virus infection fractional model. The fractional mathematical model involved the Caputo derivative. The ANN estimated the system uncertainties while the sliding mode observer eliminated the external disturbances. In this work, the finite-time stability of the observer was guaranteed. In fact, the radial basis function ANN estimated the complex nonlinearity of the system accurately.

Finally, in Ref. [24], the fractional-order Darwinian particle swarm optimization (FO-DPSO) algorithm was employed for training a feed-forward ANN to approximate the solutions of the corneal shape model for eye surgery. In this work, the Adams–Bashforth–Moulton numerical method was used to show the effectiveness of the proposed scheme

ANN involved with FC: applications in unmanned aerial vehicles (UAVs)

ANN involved with FC has been applied to unmanned aerial vehicles (UAVs). Some relevant works have been reported in Refs. [144, 307]. Following, we will address these works.

In Ref. [144], the authors approximated the realization of an analogical fractional-order PID controller using feed-forward ANN and a finite impulse response filter to establish a robust control for applications of unmanned aerial vehicles (UAVs). Posteriorly, in Mobarez et al. the authors implemented a fractional-order PID control based on ANN for fixed-wing UAVs, where the proposed autopilot was evaluated in linearized and nonlinear systems. The fractional controller showed better performance against wind disturbance, the effect of the sensors’ noise, and system uncertainties when compared with other controllers.

ANN involved with FC: applications in circuits realization

ANN involved with FC has been applied to circuit realization. In works [25, 165, 189, 198, 199, 292] relevant research have been presented on this issue. Below we will present an overview of these works:

In Ref. [165], the authors developed an adaptive fractional sliding mode control involving the Caputo derivative. The fractional controller was based on the dual radial basis function ANN, allowing to improve the performance of three-phase shunt active power filters. Also, the Chua’s circuit with Caputo derivative was developed in Ref. [25] to investigating the fractional Chua’s system and discuss its chaotic behavior. Furthermore, in Ref. [189], the authors developed an adaptive ANN backstepping control of fractional-order Chua–Hartley chaotic system, as well as an electronic circuit. In this work, the ANN was used to estimate the unknown nonlinear function, and the proposed controller was able to guarantee the stability of the closed-loop system. Otherwise, in Ref. [292], a lathe machine tool was the basis to study turning chatter vibration by means of ANN-based on Chua’s circuit and fractional-order Lorenz master/slave chaotic system. Finally, fractional-order sliding mode controllers based on recurrent ANN were developed for the current compensation and the current harmonic compensation of active power filter involving Caputo derivative in [198, 199], respectively.

Also, Liu [308] employed a reaction/diffusion cellular FANN to describe the diffusion behavior that happened on the electromagnetic field where the electrons describe nonuniform movement. In this work, the Caputo derivative was used, and the stability of the FANN has achieved thanks to an observed-based boundary control.

On the other hand, Ding et al. [309] accomplished the first sampling-controlled memristive FANN with stochastic sensor faults via an impulsive method based on Caputo derivative. This novel approach was applied to the fractional-order Chua’s circuit system, where they reached its stabilization.

Finally, Sanchez et al. [310] simulated the incremental capacity curve of an LFP battery model by training a recurrent FANN. In this work, the model-based health prognosis of the LFP battery with an accuracy that is comparable with the laboratory measurements was attained.

ANN involved with FC: applications in robotic

ANN involved with FC and Models to control Robots have been developed in Refs. [52, 68], and they will be summarized as follows:

In Ref. [52], the authors proposed a deep convolutional ANN based on the fractional-order sliding mode control scheme to control trajectory tracking of robotic manipulators involving Caputo derivative. The proposed control showed to have robust performance against parametric uncertainties and external disturbances. In this work, several simulations were carried out to validate the proposed methods.

In Ref. [68] was presented a nonlinear neural fractional-order PID controller based on ANN applied to the motion control of nonholonomic differential drive mobile robot, involving Grünwald–Letnikov derivative. The hybridization of a modified adaptive PSO and the EFF optimization algorithms were implemented for tuning the parameters of the fractional-order PID controller based on ANN. The authors showed with numerical simulations that the performance of the designed fractional controller was excellent compared with nonlinear controllers of integer-order on the trajectory tracking of the differential drive mobile robot with different trajectories as study cases.

ANN involved with FC: applications in image encryption

ANN involved with FC have been applied to image encryption in Refs.: [82, 186, 294, 295]. Such works will be overviewed as follows:

First, in Ref. [186], the authors presented an image encryption approach based on impulsive synchronization of chaotic ANN applying the fractional-order approach involving Caputo derivative and considering a delayed cellular FANN.

Second, image encryption algorithms were designed based on chaotic fuzzy cellular neural FANN with time-varying delays and chaotic three-dimensional discrete Hopfield FANN in Refs. [82, 294], respectively. The authors used the Caputo derivative and the FANNs as a pseudo-random number generator. The dynamic behavior and synchronization of the systems were investigated and applied to image encryption algorithms where solutions allowed to improve encryption security. Third, in [82], numerical evaluations were developed, and analysis of bifurcation diagrams, phase space diagrams, and time series plots to explore the effects of the time-varying delay and the fractional-order. In this work, the global stability conditions were derived under the design of an adaptive control approach guaranteeing the global asymptotic and the exponential stability by synchronizing the drive-response system when time tends to large. On the other hand, in Ref. [294], phase portraits, bifurcation diagrams, and Lyapunov exponents were developed to show the chaotic dynamics of the system. A control approach was designed to synchronize the system. The results show the effectiveness of the encryption system. The simulation results demonstrated that the algorithms have good encryption features.

Similar to the research works presented above, in Ref. [295], another chaotic image encryption algorithm was proposed. In this case, a five-dimensional cellular FANN was employed as a diffusion controller in the encryption system. The results showed that this new algorithm improves encryption efficiency with good security performance. Furthermore, it resisted the common attack methods.

Finally, the synchronization of a class of FANNs was carried out by designing an adaptive control to develop a crypto-system algorithm for encryption/decryption of unmanned aerial vehicle color images in secure communications, employing Caputo derivative [311].

ANN involved with FC: applications in other engineering applications

Blasik et al. [312] developed an accurate numerical method based on ANN and Caputo derivative to extend the front fixing method developed in a previous work based on the one phase fractional-order Stefan problem (anomalous molecular diffusion where the diffusion coefficient is generalized).

Future research topics arising on ANNs involved with FC

The main advantage of modeling systems involving FANNs is that the fractional derivatives are an excellent tool in describing the memory and hereditary properties of various processes. Therefore, the simulated systems involving FANNs are more accurate than the integer-order models. However, there are few works done in this field to date, and there are many important topics that have not been approached yet. Therefore, we would like to recommend some future research topics where the FANNs could be successfully applied, which are detailed as follows:

We propose to apply the FANNs to the identification, control, performance studies, and prediction of behaviors on physical models. Thus, the models could improve their accuracy without increasing their complexity.

The employment of FANNS for the simulation of trajectorial physical models could allow that the effectiveness of the trajectories prediction to be increased, avoiding wasting time and resources in the experimental studies.

The management of the plants of industrial processes is usually complex in nature. Often, the training ANN models must attain a fast response concerning corresponding physical models and for the real-time monitoring of the plants; We propose to use FANNs to model the industrial plants since FANNs could do the management of them more efficiently with the fractional approach on their simulations.

We propose to apply the FANNs on the sustainable energy technologies to realize a better analysis and optimization of their parameter, and thus, help environmental care.

FANNs could be an accurate tool for the study of behaviors, the prediction of performance, and the optimization of solar collectors, biomass heating systems, wind turbines, and so on.

FANNs could be an excellent tool in studying voltage prediction on PEM fuel cells with minimum time demand and good accuracy, reducing costs and avoiding extensive experiments. FANNs could lead to a better analysis of the PEM fuel cell components and better optimize their parameters to minimize the voltage losses and reach their best efficiency.

We propose to employ FANNs on modeling of the epidemic spreading of viruses that affect the health of animals and human beings.

The employment of FANNs in modeling the parameters that determine the diagnosis of COVID-19 would provide fast and accurate diagnostics of the actual COVID-19 pandemic, helping clinicians in detecting COVID-19, quantification, follow-up of the infected cases, and helping the activation of the plan actions.

The spreading prediction and analysis of the performance of common viruses such as HIV, H1N1, Dengue, Ebola, and others could be successfully modeled using FANNs, which is beneficial for preserving lives since predictions are helpful to control and prevent the spread of the viruses.

We propose to employ FANNs to model the spread of cancer cells from where they first formed to another part of the body.

The identification of cancer cells and the prediction of their movement into the human body are essential for preserving life. Thanks to the effectiveness of FANNs on modeling and predicting behaviors in different areas of science, we believe that some accurate FANNs models based on cancer cell propagation could help in its identification and prediction.

We propose to apply the FANNs for the simulation of biological processes.

FANNs could be efficiently used in the process control of biological systems, reaching a better online optimization of these systems than in the integer-order cases.

We propose to simulate probabilistic models using FANNs.

The meteorological time series prediction could be more accurate when the ANN describes the system as simulated employing FC. Guaranteeing a good prediction, the forecast of human and economic losses in front of natural disasters could be employed successfully.

We propose combining fractional-order physically-based modeling and deep learning.

The fractional deep neural networks are a tool of machine learning and artificial intelligence more powerful than integer-order deep neural networks; the FC improves the deep learning methods, reaching a better optimization of parameters of the physical models.

We propose the application of FANNs to predict economic trends.

The realization of economic models based on FANNs could help economists and business people accurately identify entrepreneurship opportunities. This approach could help governments and economists establish correct predictions of the prices of crude oil, natural gas, power, risk management, trading strategies, etc.

Conclusions

This manuscript presented a bibliographic review of fractional calculus (FC) on artificial neural networks (ANN). We have focussed on realizing a thorough investigation related to the employment of FC on ANN, the methods used to solve their fractional differential equations (FDE), the optimization algorithms employed to train these systems, the control approaches involved with them, and their main applications in different areas of science and engineering. According to the bibliographic review, the most used method to solve the FDE on ANNE involved with FC is the analytical Laplace transform method, followed by the Adams–Bashforth–Moulton method. Also, other methods have been used to solve FDE in these systems, such as the Homotopy perturbation method, the power series expansion method, Adomian decomposition method, among others. All of them have allowed establishing a comparison between the proposed schemes and the analytical or numerical solution of FDE to validate them. Regarding the optimization algorithms used for the training of ANN involved with FC, there have been developed a few algorithms with the fractional approach, indeed, just gradient descent algorithm, Back-propagation algorithm, and Darwinian particle swarm optimization algorithm have their fractional version. Nevertheless, there are lots of classical algorithms employed with the aim of training ANN involved with FC. According to the advantages of FC, most of these algorithms could be implemented with a fractional approach in the future, obtaining results very different from the obtained using classical algorithms and probably with solutions more near reality. For training of these systems, the most used derivative was the Caputo derivative, followed by the Riemann–Liouville derivative, Atangana–Baleanu in Caputo’s sense, and Grünwald–Letnikov, respectively. Otherwise, another important topic that has been presented here is the control of such systems; since several works joined control strategies with FC to guarantee synchronization and stabilization of the ANN involved with FC. Concerning the control strategies, the most reported in the literature is the feedback control, followed by adaptive control, sliding mode control, adaptive feedback control, impulsive control, and washout filter control. Therefore, we can affirm that FC is an excellent mathematical tool to be used with ANN. Also, we have confirmed that the Caputo derivative is the most applied with ANN due to its ability to describe physical systems. The Riemann–Liouville derivative is the second derivative most used on these types of systems. This derivative is applied to describe theoretical systems, while the Grünwald–Letnikov derivative is less used in these types of systems. Other interesting applications of ANN involved with FC are the approximation of functions, the dissipativity, periodicity demonstration, description of chaotic behaviors, among others. But some of the most important advances related to ANN involved with FC are their recent applications in medicine, robotic, cryptography, image processing, and sustainable energy. This implies that the research community is paying attention to the fractional calculus theory.

We are in the presence of an interesting branch of mathematics compared with integer-order calculus. The FC can better describe several processes’ memory and genetic characteristics, having unlimited memory and more degrees of freedom.

However, few applications exist until today; we consider many promising works to do shortly. Therefore, we decided to guide the researchers for the realization of future works in this field. Some of the future directions we suggested are the following: the application of FANNs on the study of trajectorial physical models for reach more accurate simulated systems and better prediction of trajectories than in integer-order cases; the employment of FANNs for the improvement of the voltage prediction in PEM fuel cells and better identification of their parameters; the simulation of industrial plants using FANNs to reach more accurate computational systems which can be managed on real-time with fast-response. In medicine and biology, we proposed implementing FANNs based on the epidemic spreading of viruses such as Covid-19, HIV, Ebola, H1N1, and others for a better analysis, diagnosis, prediction, and forecast of them. Also, we proposed to realize the simulation of cancer cells spreading into the human body to help clinicians detect, study the behavior of cancer cells, and predict their performance accurately, helping save lives. Besides, we proposed implementing FANNs on the biology process to aid the development of process control of biological systems and predict the performance of these systems, avoiding realize exhaustive failed experiments. On the other hand, we recommended applying FANNs on probabilistic meteorological models to attain accurate systems capable of making good predictions avoiding human and economic losses when natural disasters happen. Finally, but not less important, we recommended implementing the FANNs based on economic models for guiding governments and economists to make decisions based on a correct prediction of the economic phenomena. All our recommendations are based on the certainty that the accurate models reached employing FANNs avoid the employment of exhaustive repetitive experimental tests. Thus, the FANNs help the growth of science and engineering less time employing the optimum resources for their development.