Literature DB >> 27792737

A Structure-Adaptive Hybrid RBF-BP Classifier with an Optimized Learning Strategy.

Hui Wen1, Weixin Xie1, Jihong Pei1.   

Abstract

This paper presents a structure-adaptive hybrid RBF-BP (SAHRBF-BP) classifier with an optimized learning strategy. SAHRBF-BP is composed of a structure-adaptive RBF network and a BP network of cascade, where the number of RBF hidden nodes is adjusted adaptively according to the distribution of sample space, the adaptive RBF network is used for nonlinear kernel mapping and the BP network is used for nonlinear classification. The optimized learning strategy is as follows: firstly, a potential function is introduced into training sample space to adaptively determine the number of initial RBF hidden nodes and node parameters, and a form of heterogeneous samples repulsive force is designed to further optimize each generated RBF hidden node parameters, the optimized structure-adaptive RBF network is used for adaptively nonlinear mapping the sample space; then, according to the number of adaptively generated RBF hidden nodes, the number of subsequent BP input nodes can be determined, and the overall SAHRBF-BP classifier is built up; finally, different training sample sets are used to train the BP network parameters in SAHRBF-BP. Compared with other algorithms applied to different data sets, experiments show the superiority of SAHRBF-BP. Especially on most low dimensional and large number of data sets, the classification performance of SAHRBF-BP outperforms other training SLFNs algorithms.

Entities:  

Mesh：

Year:  2016        PMID： 27792737      PMCID： PMC5085025          DOI： 10.1371/journal.pone.0164719

Source DB:  PubMed          Journal:  PLoS One        ISSN： 1932-6203            Impact factor:   3.240


Introduction

In the field of pattern recognition and data mining, as typical single-layer feed-forward networks (SLFNs), radial basis function networks (RBF) have been intensively studied over the past several decades. When used for classifying problems, there are three important factors for evaluating network performance: 1) classifying accuracy, 2) network size, and 3) training time. To achieve good network performance, different optimization algorithms are used to train the RBF hidden layer, such as K-means clustering [1, 2], fuzzy C-means clustering [3, 4], fuzzy K-nearest neighbors [5], differential evolution [6, 7], and other optimization algorithms [8-12]. However, in most of these methods, the number of RBF hidden nodes is assigned a priori, which may lead to poor adaptability for different sample sets. The selection of network size is also a critical issue. If there are too few hidden nodes the network may not be able to approximate the given function, and if there are too many, the network may exhibit poor generalization performance because of over fitting. Several sequential learning algorithms have been proposed to find a proper network size [13-16]. In [17], a minimal resource allocation network (MRAN) is proposed, which is allowed to delete the previous center. The deletion strategy is based on the overall contribution of each hidden unit to the network output. A sequential learning algorithm for growing and pruning the RBF (GAP-RBF) and a generalized growing and pruning RBF (GGAP-RBF) algorithm are proposed in [18, 19], which use the significance of nodes as the learning strategy. Because the GGAP-RBF algorithm could not handle problems with high-dimensional probability density distribution, this problem is overcome in [20], which uses a Gaussian mixture model (GMM) to approximate the GGAP (GGAP-GMM) evaluation formula. In [21], an error correction (ErrCor) algorithm is used for function approximation. In each iteration of the algorithm, one RBF unit is added to fit and then eliminate the highest peak in the error surface, which can reach a desired error level with fewer RBF units. Other methods have also been established to identify a proper structure while maintaining the desired level of accuracy [22-27]. For online training algorithms, the training time is very important. This parameter directly determines how efficiently an algorithm runs. This problem is well overcome by extreme learning machines (ELMs) [28], which are also effective algorithms for training SLFNs; ELMs choose random hidden node parameters and calculate the output weights with the least squares algorithm. This method can achieve a fast training speed, as well as good classifying accuracy. In ELMs, the number of hidden nodes is assigned a priori, and many non-optimal nodes may exist; ELM tends to require more hidden nodes than conventional tuning-based algorithms [29]. Thus, in [30-33], several types of growing and pruning techniques based on ELMs are proposed to effectively estimate the number of hidden nodes. In [34], an evolutionary ELM (E-ELM) based on differential evolution and ELM is proposed; the algorithm uses the differential evolution method to optimize the network input parameters and an ELM algorithm to calculate the network output weights. Because the trial vector generation strategies and the control parameters have to be manually chosen in E-ELM, in [35], a self-adaptive evolutionary extreme learning machine (SaE-ELM) is proposed; its network hidden node parameters are optimized by the self-adaptive differential evolution algorithm, which further improves the network performance. This paper mainly focuses on how to obtain higher classifying accuracy as well as a suitable network size for the RBF hidden layer. A structure-adaptive hybrid RBF-BP (SAHRBF-BP) classifier with an optimized learning strategy is presented. SAHRBF-BP is composed of a structure-adaptive RBF network and a BP network of cascade, where the number of RBF hidden nodes is adjusted adaptively according to the distribution of sample space, and a suitable network size for RBF hidden layer that matches the complexity of the sample space can be built up. Thus, SAHRBF-BP solves the problem of dimension change from sample space mapping to feature space. In SAHRBF-BP, the nodes in the RBF network are used for nonlinear kernel mapping, the complexity of sample space is mapped onto the dimension of the BP input layer, and the BP network is then used for nonlinear classification. The nonlinear kernel mapping can improve the separability of sample spaces, and a nonlinear BP classifier can then supply a better classification surface. In this manner, SAHRBF-BP combines the local response characteristics of the RBF network with the global response characteristics of the BP network, which simplifies the selection of parameters in the BP hidden layer while reducing the dependence on space mapping in the RBF hidden layer; thus, the classification accuracy is improved while the generalization performance is guaranteed. An optimized learning strategy is presented to construct SAHRBF-BP classifier. The optimized learning strategy uses global information of training sample space and generates RBF hidden nodes incrementally. On the one hand, many optimization algorithms, such as K-means clustering, fuzzy C-means clustering, differential evolution, also use global information of training sample space to optimize RBF hidden nodes, however, the number of hidden nodes in these optimization algorithms needs to be manually determined, which may lead to poor adaptivity for different sample sets. On the other hand, the sequential learning algorithms, such as MRAN, GAP-RBF, can achieve the estimation of RBF hidden nodes for different sample sets, however, the loss of global information may lead to a reduction in classification performance. In addition, unlike GAP-RBF, the presented method does not require an assumption that the input samples obey a unified distribution. Unlike GGAP-GMM, it does not need to fit the input sample distribution. By using a potential function clustering approach to measure the density in each class of training sample space, the corresponding RBF hidden nodes that cover different sample regions can be established. It reduces the restrictions on the sample sets and is adaptable to more complex sample sets. Once an initial RBF hidden node is generated, a form of heterogeneous samples repulsive force is designed to further optimize the hidden node parameters. For each initial hidden node, in a certain region, we assume the heterogeneous samples can affect the center; that is, there is an repulsive force that makes the current center move away from the heterogeneous samples. When the center reaches a suitable position, the repulsive force will disappear. Then, a suitable width parameter can be determined accordingly. A mechanism for eliminating the potentials of the original samples is then presented. This mechanism is ready for the next learning step. Thus, the RBF centers and the width and number of RBF hidden nodes can be effectively estimated. Once the number of RBF hidden nodes is generated adaptively and the node parameters are optimized, the number of subsequent BP input nodes can be determined, and the overall SAHRBF-BP classifier is built up; then different training sample sets are used to train the BP network parameters in SAHRBF-BP, where the BP network parameters are optimized by the existing BP algorithm. In this paper, the performance of SAHRBF-BP is compared with that of other well-known training SLFNs algorithms, such as back propagation based on stochastic gradient descent(SGBP) [36], MRAN, SVM, ELM, and SaE-ELM on 108 benchmark data sets. To measure the unique features of SAHRBF-BP, the RBF nerwork based on k-means clustering (KMRBF) [2], GAP-RBF and the k-means clustering learning algorithm based on the hybrid RBF-BP network (KMRBF-BP) are added to compare with SAHRBF-BP on two artificial data sets. Experiments show that the superiority of SAHRBF-BP. Especially on most low dimensional and large number of data sets, the classification performance of SAHRBF-BP outperforms other training SLFNs algorithms.

Methods

SAHRBF-BP classifier

SAHRBF-BP is composed of a structure-adaptive RBF network and a BP network of cascade, where the number of RBF hidden nodes are adjusted adaptively according to the distribution of sample space, the complexity of sample space is mapped onto the dimension of the BP input layer, and the BP network is then used for nonlinear classification. The nonlinear kernel mapping can improve the separability of sample space, and a nonlinear BP classifier can then supply a better classification surface. To clarify the situation, Fig 1(A) and 1(B) show an illustrative diagram of sample space mapping onto feature space for different classification problems. Note that in Fig 1(A), the samples in the red box far away from the center of each kernel function will be mapped near the origin of the coordinate plane, this problem will be overcome by the optimized learning strategy presented in the next section. In Fig 1(B), with the increase in sample space complexity, the dimension of the feature space is increased accordingly, which is classified by a BP network.
Fig 1

Illustrative diagram of the sample space mapping onto feature space for different sample sets.

(A) The mapping dimension is 3 (B) The mapping dimension is 6.

Illustrative diagram of the sample space mapping onto feature space for different sample sets.

(A) The mapping dimension is 3 (B) The mapping dimension is 6. SAHRBF-BP is shown in Fig 2, which consists of four components:
Fig 2

SAHRBF-BP classifier.

The input layer, which consists of t source neurons, where t is the dimensionality of the input vector. The RBF hidden layer, which consists of a group of Gaussian kernel functions: where μ and σ are the center and width of the hidden node, respectively, and K is the number of hidden neurons. The BP hidden layer, which consists of the neurons between the RBF hidden layer and output layer. The induced local field for node j in layer l of the BP is where is the output signal of the neuron i in the previous layer l − 1 of the BP network and is the synaptic weight of neuron j in layer l that is fed from neuron i in layer l − 1. Assuming the use of a sigmoid function, the output signal of neuron j in layer l is where a and b are constants. If neuron j is in the first BP network hidden layer, i.e., l − 1, set where g(x) is the double polar output of φ(x) and can be denoted as The output layer. Set L is the depth of the BP network, note the depth of the BP network is equal to the sum of the BP network input layer, the hidden layer, and the output layer, i.e., if l = 1, then L = 3, and the output can be given as In Fig 2, the double polar processing can ensure the validity of the BP network input. In addition, the combination of the structure-adaptive adjustment of the RBF hidden layer with the BP network can provide a good complementary effect. On the one hand, the RBF network has good stability, where the activation response in the RBF hidden nodes has local characteristics and maps the output to a value between 0 and 1. Thus, the original samples, including outliers, will be limited to a finite space, and the adaptive adjustment of RBF hidden nodes can ensure the validity of the space mapping. Processing the results of mapping the RBF hidden nodes and using them for the input of the BP network can reduce the dependence on the selection of BP network parameters; furthermore, the convergence rate of the BP network can be increased and local minima can be avoided. On the other hand, in a BP network, the activation response in hidden nodes has global characteristics, especially those regions not fully displayed in the training set. In SAHRBF-BP, the BP network is used for nonlinear classification, which can reduce the dependence on the original sample space mapping. Even if there are errors in the original sample space mapping, the nonlinear BP network can be compensated for to a certain extent. Therefore, SAHRBF-BP combines the stability of the RBF network with the generalization ability of the BP network and improves the classification performance further. A single hidden layer multilayer perceptron neural network with input-output mapping can provide an approximate realization of any continuous mapping [37]. In light of the foregoing discussion, in SAHRBF-BP, we set the number of BP hidden layers to l = 1, and the number of hidden nodes of BP should be appropriately increased with the increase in the complexity of the sample space. When SAHRBF-BP classifier is built up, however, new problems may arise because the number of RBF hidden nodes and their parameters are unknown, and inappropriate kernel mapping will deteriorate the network performance. This process can be overcome by the optimized learning strategy presented in the next section.

The optimized learning strategy

Main objective

To obtain good classifying performance for a given training sample set, it is necessary to fully use training sample information. Fig 3 demonstrates such a scenario, where the generated RBF hidden nodes are used to cover samples of class 2. However, these RBF hidden nodes may cover samples of classes 1 and 3, which leads to a reduction in classification performance. Our main objective is to design a method that can optimize the coverage of each class of training samples, where each coverage generates a RBF hidden node and ultimately estimates the center, the width and the number of RBF hidden nodes. For that purpose, the following issues should be considered:
Fig 3

Example of the current RBF hidden nodes covering other classes of samples.

To optimize the coverage of the training sample space, a suitable initial RBF hidden node must be established each time. The adjustments of the center and width should meet certain criteria such that each generated RBF hidden node can cover the samples of the current class as much as possible, while covering the samples of other classes as little as possible. To address issue 1), we consider that for each class of training samples, in different regions, their densities are different. To cover the training sample space effectively, the sample in the most intensive region can be selected as the initial center. Therefore, it is necessary to quantify each class of samples and establish a standard for measuring the density of the input sample space. In this paper, a potential function is introduced into training sample space. By calculating the sample potentials in each class, the densities of different regions can be measured, where the sample with the maximum potential value can be used as the initial center. To address issue 2), we consider that the information of other classes of samples can be used to adjust the center and width such that an optimization model is established, where a form of heterogeneous samples repulsive force is designed to adjust the center and the width adaptively. To complete the main objective, the following steps can be followed. Compute the potential value of each sample in the current class. Set the sample with the maximum potential value as the initial center. Consider the distance between the heterogeneous samples and the center; in a certain region, the center and width should be adjusted adaptively by a form of heterogeneous samples repulsive force. Eliminate the potential value of each sample in the current class. Iterate Steps 2-4 until the stop condition is met, then turn to learn other classes of samples.

Algorithm principle

In the field of pattern recognition, potential functions can be used for density clustering and image segmentation (IS). Several methods of constructing potential function are proposed in [38]; here, we choose the potential function where γ(x, x) represents the interaction potential of two points x, x in the input sample space, d(x, x) represents the distance measure, and T is a constant, which can be regarded as the distance weighting factor. Given a training sample set S, where a specific label y, y ∈ {y; i = 1, 2, … h} is attached to each sample vector x in S, h is the number of pattern class. Let S denote the set of feature vectors that are labeled y, , where N is the number of training samples in the ith pattern class. Thus , S ∩ S = ∅, ∀i ≠ j. For a pair of samples in S, its interaction potential can be denoted as Let be the baseline sample; therefore, the interaction potential of all other samples to can be denoted as Once the potentials of each sample in S are given, the sample with the maximum potential can be selected, where it is assumed the sample is , that is, To generate valid Gaussian kernel functions, we find the densest region in the sample space and then establish a Gaussian kernel to cover the region. To that end, the sample with the maximum potential is chosen as the initial center of the Gauss kernel function, which is expressed as follows: where k refers to the number of RBF hidden neurons generated. Once the width is given, an initial RBF hidden node is established, which can be used to cover samples of the current class. However, the generated RBF node takes into account sample information about the current class only, which may cause the current RBF hidden node to cover samples of other classes. To achieve the optimization coverage of each class of training samples, here, heterogeneous samples are taken into account to optimize the initial hidden node parameters. A form of heterogeneous samples repulsive force is used to adjust the center and the width adaptively. To make the center adjustment, first, the direction of each heterogeneous sample repulsive force should be in line with the centerline, which can make the center far away from the heterogeneous sample directly. Second, when a heterogeneous sample is close to the center, the magnitude of the center should be adjusted by a large margin, whereas when a heterogeneous sample is relatively far from the center, the magnitude of the center should be adjusted slightly. According to the foregoing description, the heterogeneous sample repulsive force is defined as follows: Definition: Given two vectors, where one is the center and the other is a heterogeneous sample, there is a repulsive force that makes the heterogeneous sample point to the center. The magnitude of the force is inversely proportional to the square of the distance between the two vectors, and the direction of the repulsive force is in line with the centerline. To adjust the center by the form of heterogeneous sample repulsive force, two hypothetical conditions should be met: When the initial center is determined, given the initial width, only in the current coverage region, the heterogeneous sample repulsive force exists. When the center is adjusted to a suitable position, the heterogeneous sample repulsive force will disappear. Condition 1) demonstrates the case when the distance between the center and the heterogeneous samples is outside a certain range, the heterogeneous sample repulsive force can be ignored; this condition simplifies the study of the problem. For condition 2), the key is to establish criteria to make the center converge toward a suitable position. According to the definition of the heterogeneous sample repulsive force and condition 1), the optimization model can be given as follows. Given the initial width σ, when the initial center is generated, where . For a heterogeneous sample , , if , where λ is a width covering factor, there is a repulsive force from to , which can be denoted as Here, a negative exponential function is chosen to express the relationship between the heterogeneous sample repulsive force and the distance: where α is a positive constant and can be seen as the heterogeneous sample repulsive force control factor. Assume the number of heterogeneous samples in the current covered region is M. Adding all the heterogeneous samples repulsive force, the center can be adjusted as follows: Note that for a sample x, when x ∉ S, if |x − || < λ ⋅ σ, M counts plus 1. Similarly, set M denotes the number of samples in the current class in the current covered region, when x ∈ S, if ||x − || < λ ⋅ σ, M counts plus 1. Fig 4 provides a geometrical description of the heterogeneous sample repulsive force model, where the black and red boxes denote the covered region before and after the center adjustment, respectively. In Fig 4, there is a repulsive force makes samples 1 and 2 point to the initial center. The resultant force adjusts the initial center to a new position that is far away from samples 1 and 2.
Fig 4

Model of heterogeneous sample repulsive force.

For condition 2), to make the center reach a suitable position, it is necessary to carry out multiple iterations. Set M is the iteration step variable, at the initial stage, the magnitude of the center adjustment is relatively large; with the increase in the iteration step, the magnitude of the center adjustment will gradually decrease and eventually converge to a suitable position. To ensure validity of the center adjustment, set and represent the number of current class of samples and heterogeneous samples covered in the updated region, respectively, for each center adjustment, Eq (14) can be corrected as follows: In practice, because of the complexity of different training sample sets, even if the center is adjusted to a suitable position, the generated RBF hidden nodes may also cover heterogeneous samples under the given initial width parameters. Decreasing the width may be one way of reducing the coverage of heterogeneous samples; however, if the width is too small, the generalization performance will be greatly reduced. To further complete the optimization coverage of different regions of the each class of training samples, and guarantee good generalization performance, the width adjustment is set as follows: where σmin is the minimum width, β is a fixed value and can be seen as the width constraint factor. In practice, to ensure validity of the width adjustment, the selection of β should be a little less than the width covering factor λ and the adjusted width should be in the range between σmin and σ. For different RBF hidden nodes, this adjustment can ensure the relative difference of the widths to better fit the training sample space. Note that the adjustment of the width is carried out only once, after the center adjustment. When a hidden node is established, it is necessary to eliminate the potentials of the region to find the next initial center in the remaining samples. This process can be accomplished as follows: where is the initial center of the current hidden neuron. For the potential value update process, Eq (17) shows when a sample is close to the initial center , the potential value of is attenuated fast, whereas when a sample is far away from the center, the potential value of is attenuated slowly. When meeting the inequality the learning process goes on and is ready to search the next initial center. Otherwise, the algorithm of constructing RBF hidden nodes in the current pattern class is terminated and turns to learn other pattern classes, where δ is a threshold. Fig 5 shows the illustrative diagram of adaptively generating RBF hidden nodes and optimizing node parameters, where the number of RBF hidden nodes is increased incrementally, and each initial RBF hidden node is determined by a potential function clustering approach, then a form of heterogeneous sample repulsive force is used to further optimize each RBF hidden node parameters. In Fig 5, the black line box represents regions covered by the initial center and the width, the red line box represents the final coverage regions.
Fig 5

Illustrative diagram of adaptively generating RBF hidden nodes and optimizing node parameters.

Combining SAHRBF-BP classifier, the optimized learning strategy is summarized in Algorithm 1. Algorithm 1 The optimized learning strategy Initialization; for i = 1: c    % for each class of training samples Compute the potential value of each sample according to Eq (9). while Determine the maximum potential value of each sample according to Eq (10). The number of RBF hidden nodes counts plus 1, allocate an initial center using Eq (11). Determine M, M. Use Eqs (13) and (14) to update the initial center, determine , . while if , . Use Eq (15) to update the center, update , . M ← M + 1. else Use Eq (16) to update the width. Break; end if end while Eliminate the sample potential value of the region according to Eq (17). end while end for Use Eqs (1) and (5) to compute g(x), let g(x) be the input of the BP network, where g(x) = (g1(x), g2(x), …, g(x)). while ||e|| > mse_thres && m ≤ num_Epoch Use Eqs (2)–(4) and Eq (6) to compute the error signal e = d − o, where d is the jth element of the desired response vector d. Compute the local gradients of the network as follow where is the differentiation with respect to the argument. Adjust the synaptic weights of the network in layer l of BP as below. where τ is the momentum constant, η is the learning rate. m ← m + 1. end while

Adjustment of the output label values

The SAHRBF-BP algorithm can handle binary class problems and multi-class problems. For multi-class classification problems, suppose that the observation dataset is given as , where x ∈ R is an t–dimentional observation features and y ∈ R is its coded class label. Here, h is the total number of classes, which is equal to the number of output hidden neurons. If the observation data x is assigned to the class label c, then the cth element of y = [y1, …, y, … y] is 1 and other elements are -1, which can be denoted as follows: The output tags of SAHRBF-BP are , where According to the coding rules, only one output tag value is 1, and the other value is -1. If this condition is not met, the output tag is saturated and must be adjusted. Therefore, we set an effective way to correct the saturation problem in the learning process, which can be denoted as the pseudo code in Algorithm 2. Algorithm 2 The method of adjusting the output saturation problem Given observation dataset , for every input vector x, while j ≤ h if the number of is equal to h Set max(o) = 1 and and hold other output values fixed. end if if the number of is more than 1 Set max(o) = 1 and the other output values are -1. end if end while

Results

In this section, we evaluate the performance of SAHRBF-BP using two artificial data sets and 108 benchmark data sets, where Double moon data set are taken from [39], 101 benchmark data sets are taken from the UCI machine learning repository [40]. In addition, seven benchmark data sets including cod_rna, DNA, fourclass, ijcnn1, splice, svmguide1 and svmguide3 are taken from [41]. Tables 1–3 provide descriptions of the benchmark data sets. The benchmark data sets are grouped into three categories: binary class, multi-class and large number of samples. All benchmark binary class and multi-class benchmark data sets are grouped into low dimensional and high dimensional sets. For all benchmark data sets, the inputs to each algorithm are scaled appropriately to fall between -1 and +1. In each data set, the training set, validation set and testing set are independent. For balanced data sets, the number of training samples in each class is identical, which is also applicable to the validation set and testing set. For imbalance data sets, when the number of training samples is given, the number of each class of training samples is determined according to the proportion of each class in the whole data set. This method is also applicable to the validation set and testing set.
Table 1

Descriptions of binary class data sets.

IndexData sets# Classes# Features# Training# Validation# Testing
B01Australian214345172173
B02Banknote24686343343
B03Blood24374187187
B04Breast cancer291387069
B05Breast(original)29350174175
B06Cleve2131487474
B07Diabetes2857610092
B08Fertility29503020
B09Fourclass22432116115
B10Haberman231537677
B11Heart disease2131517676
B12Liver261728786
B13Mammographic25480241240
B14Monk126124100332
B15Monk226169100332
B16Monk326122100332
B17Planning212914645
B18Svmguide124308915002500
B19Vertebral261055253
B20Wholesale27220110110
B21Wilt2630001339500
B22Breast(diagnostic)230284143142
B23Breast(prognostic)2331004949
B24Chronic224200100100
B25Climate218270135135
B26Congressional216210100125
B27First order2513059800730
B28Hill2100606303303
B29Hill(with noise)2100606303303
B30German224500250250
B31Ionosphere2341758888
B32LSVT2310633231
B33Mushrooms221300015003624
B34Musk12167238119119
B35Musk22167320016001598
B36Parkinsons220974949
B37QSAR241527264264
B38Retinopathy219575288288
B39Secom2591783392392
B40Seismic bumps2181292646646
B41Sonar2601045252
B42Spambase257230011511150
B43Spect heart2228047140
B44Splice26010005001675
B45Svmguide3222100024341
B46Vote216217109109
Table 3

Descriptions of large number of samples data sets.

IndexData sets# Classes# Features# Training# Validation# Testing
L01A1a(adult)214160595730000
L02A6a(adult)21411221500016342
L03Action1(normal)108200001000068886
L04Action2(normal)108150001000074930
L05Action3(normal)108150001000074872
L06Action4(normal)108150001000074069
L07Action1(aggressive)108150001000073172
L08Action2(aggressive)108150001000074690
L09Action3(aggressive)10810000500089543
L10Action4(aggressive)10810000500082764
L11Action1(abnormal detection)28150005000177058
L12Action2(abnormal detection)282000010000169620
L13Action3(abnormal detection)282000015000179415
L14Action4(abnormal detection)282000010000166833
L15Cod_rna28152201000020000
L16Credit22310000500015000
L17Eye214749037453745
L18Gas(2013)6128691035003500
L19Ijcnn121315000500015000
L20Letter26161000050005000
L21Occupancy25814326659752
L22Pendigits1016749520001498
L23Record27200001000094913
L24Sensorless114811000770039809
L25Skin232500010957210000
L26Shuttle59335001000014500
L27Telescope210951050004510
The performance of SAHRBF-BP is compared with other well-known training and optimization SLFNs algorithms, such as SGBP, MRAN, SVM, ELM, and SaE-ELM on different data sets. To measure the unique features of SAHRBF-BP, other optimization algorithms such as KMRBF, GAP-RBF and KMRBF-BP are also compared to SAHRBF-BP on two artificial data sets, namely Double moon and Concentric circle. For SGBP, the momentum constant is set to τ = 0.1. For SVM, the RBF is used as the kernel function, the cost C is selected from the set [212, 211, …, 1] and kernel parameter is selected from the set [2−3, 2−2, …, 24]. For GAP-RBF and MRAN, the common parameters are fixed to εmax = 0.5, εmin = 0.01, k = 0.8 and γ = 0.09. Other parameters for GAP-RBF are set to emin = 0.01; for MRAN, the parameters are set to emin = 0.5, , the sliding window M is selected from the set [30, 50, 100, 200, 400]. For SaE-ELM, the number of populations NP is selected from the set [20, 50, 100, 200, 500]. For SAHRBF-BP, the common parameters of the distance weighting factor, width covering factor, width constraint factor, potential value learning threshold are set to T = 1, λ = 1.5, β = 1.3, δ = 0.001, respectively. The heterogeneous sample repulsive force control factor α is selected from the set [2, 5, 10, 15, 20]. The initial width σ is selected from the set [0.4, 0.5, …, 1.6]. Note the number of hidden nodes in KMRBF, KMRBF-BP and SaE-ELM is selected manually. When gradually increasing the number of hidden nodes, the one with the lowest overall validation error is selected as the number of hidden nodes. For benchmark data sets with the number of training samples is less than 2000 and artificial data sets, simulations in each algorithm are performed 20 times and are conducted in the MATLAB 2013a environment on an Intel(R) Core(TM) i5 with a 3.2GHZ CPU and 4G of RAM. For other data sets, simulations in each algorithm are performed 3∼15 times and are conducted in the MATLAB 2013a environment on an Intel(R) Xeon(R) CPU E5-2687w @3.40GHZ(dual processor) and 128G of RAM. For each algorithm, the one with the lowest validation error is used to determine the parameters in the training models. The simulations for the SVM are carried out using the popular LIBSVM package in C [41].

Performance measures

In this paper, the overall and average per-class classification accuracies are used to measure performance. Class-level performance is measured by the percentage classification (η), which is defined as where q is the number of correctly classified samples and is the number of samples for the class y in the training/testing data set. The overall classification accuracy(η) and the average per-class classification accuracy(η) are defined as where h is the number of classes, N is the number of training/testing samples. Thus, for balanced classification problems, the overall testing η is used to measure the performance of each algorithm. For imbalanced classification problems, the overall testing η and the average testing η are used to measure the performance of each algorithm.

Performance comparison

Artificial binary class data set: The Double moon classification problem

The Double moon data set and the classifying results of SAHRBF-BP are shown in Fig 6(A) and 6(B), respectively. The classification results illustrate that SAHRBF-BP can provide a superior classification surface. Fig 7(A)–7(C) show when the initial width takes different values, the adaptive coverage of the training sample space can be completed effectively. Each cover generates a RBF hidden node and the number of RBF hidden nodes is increased incrementally, the bold lines represent the first coverage region in each pattern class, which can be seen as the densest region learned by the potential function clustering approach. With the change of the initial width, the number of RBF hidden nodes and node parameters are changed accordingly. Based on the potential function clustering to generate initial RBF hidden nodes, the form of heterogeneous sample repulsive force can further ensure that each initial RBF hidden node adjusted to a suitable position. In this way, the optimal coverage of the training sample space can be completed and the RBF centers and the width and number of RBF hidden nodes can be effectively estimated.
Fig 6

Double moon classification problem.

(A) Double moon data set (B) Classifying result of SAHRBF-BP.

Fig 7

Using different width parameters to cover the training sample space for Double moon classification problem.

(A) σ = 2 (B) σ = 3 (C) σ = 4.

Double moon classification problem.

(A) Double moon data set (B) Classifying result of SAHRBF-BP.

Using different width parameters to cover the training sample space for Double moon classification problem.

(A) σ = 2 (B) σ = 3 (C) σ = 4. In Fig 8, when the number of training samples has changed, KMRBF-BP needs less number of RBF hidden neurons than KM-RBF and can get a higher classifying accuracy. This results demonstrate the hybrid RBF-BP network structure is effective, which can improve the classifying accuracy and reduce the dependence on the original sample space mapping. Note the number of KM-RBF and KMRBF-BP is selected manually, when changing the number of hidden neurons several times, the one with the highest overall validation accuracy is selected as the suitable number of hidden neurons. The classifying accuracy of SAHRBF-BP is comparable with KMRBF-BP, however, the number of RBF hidden nodes in SAHRBF-BP is generated adaptively. The classifying accuracy of SAHRBF-BP outperforms SGBP apparently, which further shows the effectiveness of SAHRBF-BP, the reason is that the structure-adaptive RBF network can improve the separability of sample spaces. Compared to GAP-RBF, SAHRBF-BP can better adapt to the change of sample space. The classifying accuracy of SAHRBF-BP outperforms GAP-RBF and needs less RBF hidden nodes. The classifying accuracy of SAHRBF-BP is comparable with SVM, however, the number of RBF hidden nodes in SAHRBF-BP is less than SVM apparently. Thus, SAHRBF-BP can adapt training sample space well, which can get a high classifying accuracy, as well as a compact network size for the RBF hidden layer.
Fig 8

Performance comparisons between SAHRBF-BP and other algorithms on Double moon data set.

(A) Number of training samples- Number of RBF hidden neurons/ support vectors (B) Number of training samples- Overall classifying accuracy.

Performance comparisons between SAHRBF-BP and other algorithms on Double moon data set.

(A) Number of training samples- Number of RBF hidden neurons/ support vectors (B) Number of training samples- Overall classifying accuracy.

Artificial binary class data set: The Concentric circle classification problem

The Concentric circle data set and the classifying results of SAHRBF-BP are shown in Fig 9(A) and 9(B), respectively. Compared to the Double moon classification problem, the Concentric circle classification problem is more complex and can thus be used to measure the unique features of SAHRBF-BP. The classification results illustrate SAHRBF-BP can still provide a superior classification surface for Concentric circle classification problem.
Fig 9

Concentric circle classification problem.

(A) Concentric circle data set (B) Classifying result of SAHRBF-BP.

Concentric circle classification problem.

(A) Concentric circle data set (B) Classifying result of SAHRBF-BP. Fig 10(A)–10(C) show when the initial width takes different values, the adaptive coverage of the training sample space can be completed effectively. Each cover generates a RBF hidden node and the number of RBF hidden nodes is increased incrementally, the bold lines represent the first coverage region in each pattern class. With the change of the initial width, the number of RBF hidden nodes and node parameters change accordingly. For each generated initial RBF hidden node, the form of heterogeneous sample repulsive force can further ensure each initial RBF hidden node adjusted to a suitable position. Thus, the optimal coverage of the training sample space can be completed and the RBF centers and the width and number of RBF hidden nodes can be effectively estimated.
Fig 10

Using different width parameters to cover the training sample space for Concentric circle classification problem.

(A) σ = 0.1 (B) σ = 0.2 (C) σ = 0.3.

Using different width parameters to cover the training sample space for Concentric circle classification problem.

(A) σ = 0.1 (B) σ = 0.2 (C) σ = 0.3. Fig 11(A) and 11(B) demonstrate when the number of training samples changes, KMRBF-BP needs less number of RBF hidden neurons than KM-RBF and can get a higher classifying accuracy. Thus the hybrid RBF-BP network architecture improves the classifying accuracy and reduces the dependence on the original sample space mapping. Note in KM-RBF and KMRBF-BP, when the number of training samples changes, the number of RBF hidden neurons has to be adjusted manually, otherwise it will lead to a poor classification accuracy. Compared with KM-RBF and KMRBF-BP, SAHRBF-BP can adapt the training sample space well, when the number of training samples changes, the number of RBF hidden neurons in SAHRBF-BP changes accordingly, and can get a higher classifying accuracy. Compared to GAP-RBF, SAHRBF-BP can better adapt to the change of sample space. The classifying accuracy of SAHRBF-BP outperforms GAP-RBF and ELM apparently. When the number of training samples is more than 500, the classifying accuracy of SAHRBF-BP outperforms SVM. In this way, the effectiveness of SAHRBF-BP is further verified.
Fig 11

Performance comparisons between SAHRBF-BP and other algorithms on Concentric circle data set.

(A) Number of training samples- Number of RBF hidden neurons/ support vectors (B) Number of training samples- Overall classifying accuracy.

Performance comparisons between SAHRBF-BP and other algorithms on Concentric circle data set.

(A) Number of training samples- Number of RBF hidden neurons/ support vectors (B) Number of training samples- Overall classifying accuracy. For the Concentric circle classification problem, SAHRBF-BP can get a higher classifying accuracy than other training SLFNs algorithms, however, there are still a certain number of incorrect predictions in SAHRBF-BP. In Fig 9(B), we can see the samples of incorrect predictions generally appear in the boundary region. A main reason is that, due to the complexity of the sample set, it is often difficult to achieve an ideal coverage of the sample space. As shown in Fig 10(A)–10(C), if the current class of RBF hidden nodes cover heterogeneous samples, it may lead to a reduction in classification performance. From this point of view, to get a higher classification performance, it is necessary to optimize each generated RBF hidden node. In SAHRBF-BP, the combination of potential function clustering and heterogeneous samples repulsive force can adaptively determine the number of RBF hidden nodes and optimize node parameters, which ensures each generated RBF hidden node covers the samples of the current class as much as possible, while covering heterogeneous samples as little as possible. In Fig 11(B), when the number of training samples is reduced, it will lead to a reduction in classification performance. Especially when the number of training samples is 200, the overall classifying accuracy of SAHRBF-BP is a little lower than SVM. Fig 12 further shows inadequate training samples lead to the reduction of the classification accuracy. For complex data sets, when the number of training samples is reduced, the randomness of training samples in the sample space is enhanced, which can not effectively reflect the actual distribution of the entire data set, and may lead to some extent of failure by the methods of potential function clustering and heterogeneous samples repulsive force.
Fig 12

The learning effect of training and testing sample space when the number of training samples is 200.

(A) Covering effect of training sample space (B) Classifying result of SAHRBF-BP.

The learning effect of training and testing sample space when the number of training samples is 200.

(A) Covering effect of training sample space (B) Classifying result of SAHRBF-BP.

Benchmark binary class classification problems

In this section, 21 benchmark binary class low dimensional data sets and 25 benchmark binary class high dimensional data sets are used to evaluate the performance of SAHRBF-BP. Fig 13 shows the overall testing accuracy comparisons between SAHRBF-BP and other learning algorithms. For binary class low dimensional data sets, the overall testing accuracy of SAHRBF-BP is higher than other learning algorithms on most data sets, except for Fertility(B08), Haberman(B10) and Planning(B17) data sets. For binary class high dimensional data sets, the overall testing accuracy of SAHRBF-BP is higher than other learning algorithms on Breast(diagnostic)(B22), Chronic(B24), Climate(B25), Congressional(B26), First order(B27), German(B30), Ionosphere(B31), Retinopathy(B38), Spambase(B42) and Vote(B46) data sets. The overall testing accuracy of SAHRBF-BP is comparable with SVM on Mushrooms(B33), Musk2(B35), Seismic bumps(B40), Splice(B44) and Svmguide3(B45) data sets, however, the overall testing accuracy of SAHRBF-BP is lower than SVM on Musk1(B34), Parkinsons(B36), QSAR(B37), Secom(B39) and Sonar(B41) data sets, lower than ELM and SaE-ELM on Hill(with noise)(B29) data set, lower than SVM, ELM and SaE-ELM on Hill(B28) data set, and lower than SGBP and SVM on Breast(prognostic)(B23) and Spect heart(B43) data sets.
Fig 13

Overall testing accuracy comparisons between SAHRBF-BP and other algorithms on benchmark binary class data sets.

(A) Binary class low dimensional data sets (B) Binary class high dimensional data sets.

Overall testing accuracy comparisons between SAHRBF-BP and other algorithms on benchmark binary class data sets.

(A) Binary class low dimensional data sets (B) Binary class high dimensional data sets. Tables 4–6 give performance comparisons between SAHRBF-BP and other learning algorithms, where a few cases of success and failures are given a more detail description. In Tables 4 and 5, the overall and average testing accuracies of SAHRBF-BP are clearly higher than SGBP. For Blood data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.8%, and SaE-ELM, ELM, MRAN by approximately 1.5%-3.3%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.2%, and SaE-ELM, ELM, MRAN by approximately 1.4%-4.5%.
Table 4

A few cases of success in SAHRBF-BP compared with other learning algorithms on benchmark binary class data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
B03BloodSGBP43.2948.747
MRAN75.8373.21177
ELM76.4875.3280
SaE-ELM77.6476.2840
SVM77.2775.53265a
SAHRBF-BP79.1277.69113&7b
B07DiabetesSGBP38.6252.587
MRAN75.3974.7341
ELM78.6478.3650
SaE-ELM80.1379.7222
SVM80.6377.41301a
SAHRBF-BP82.3782.6519&4b
B11Heart diseaseSGBP48.3748.768
MRAN78.1677.9429
ELM79.5678.6230
SaE-ELM82.5282.4220
SVM81.7085.8542a
SAHRBF-BP83.1383.6412&4b
B13MammographicSGBP45.6553.818
MRAN77.4975.2886
ELM80.5479.1550
SaE-ELM81.3280.4230
SVM82.6482.28196a
SAHRBF-BP84.5283.7426&6b
B14Monk1SGBP49.7748.318
MRAN78.5177.4232
ELM79.4077.8430
SaE-ELM82.5281.2820
SVM87.5086.2156a
SAHRBF-BP88.6487.8323&8b
B15Monk2SGBP33.3537.737
MRAN72.8670.6658
ELM73.1272.5850
SaE-ELM75.1574.5730
SVM82.8783.1472a
SAHRBF-BP84.2183.6534&6b
B18Svmguide1SGBP49.9862.6320
MRAN86.5685.16621
ELM90.5290.36200
SaE-ELM91.4990.82160
SVM91.1590.742096a
SAHRBF-BP92.6292.28442&9b
B19VertebralSGBP70.7476.426
MRAN80.4378.5134
ELM82.2581.4640
SaE-ELM83.7283.1930
SVM84.1283.8071a
SAHRBF-BP86.4185.2345&6b

aSupport vectors.

bRBF&BP hidden nodes.

Table 6

A few cases of failures in SAHRBF-BP compared with other learning algorithms on benchmark binary class data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
B08FertilitySGBP89.5086.527
MRAN79.0077.3625
ELM82.5081.2840
SaE-ELM83.0081.8520
SVM84.0082.9418a
SAHRBF-BP80.5078.4211&4b
B17PlanningSGBP71.3970.216
MRAN67.2465.8624
ELM65.4766.2330
SaE-ELM66.5266.5820
SVM71.4370.5269a
SAHRBF-BP68.7667.3424&5b
B28HillSGBP51.6550.5210
MRAN52.3851.61226
ELM55.9154.32120
SaE-ELM56.5355.8380
SVM55.9454.76551a
SAHRBF-BP53.6552.67243&8b
B32LSVTSGBP33.3346.517
MRAN71.5869.4238
ELM73.1472.4150
SaE-ELM74.5572.8720
SVM85.7986.6153a
SAHRBF-BP72.5271.8612&4b
B34Musk1SGBP43.2843.799
MRAN75.5774.8793
ELM74.7974.5260
SaE-ELM75.8175.2530
SVM88.0388.26147a
SAHRBF-BP79.2678.82117&7b
B37QSARSGBP46.7144.538
MRAN82.2780.72107
ELM84.6285.2880
SaE-ELM85.6485.8060
SVM87.1686.52192a
SAHRBF-BP85.7184.8543&5b
B41SonarSGBP46.0947.178
MRAN67.2665.2974
ELM70.3770.0650
SaE-ELM70.2469.6230
SVM80.8584.9746a
SAHRBF-BP77.7476.5949&5b
B43Spect heartSGBP91.9890.218
MRAN66.7265.6332
ELM66.2465.5650
SaE-ELM68.0367.8320
SVM70.0572.5278a
SAHRBF-BP67.5865.8414&4b

aSupport vectors.

bRBF&BP hidden nodes.

Table 5

A few cases of success in SAHRBF-BP compared with other learning algorithms on benchmark binary class data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
B20WholesaleSGBP76.2774.588
MRAN87.5585.2141
ELM88.9287.3160
SaE-ELM90.4189.4530
SVM90.8490.4255a
SAHRBF-BP92.1791.5436&5b
B25ClimateSGBP91.4890.627
MRAN88.2686.6827
ELM91.8591.5350
SaE-ELM92.5191.9430
SVM92.3292.6449a
SAHRBF-BP93.4792.4113&5b
B26CongressionalSGBP48.5352.797
MRAN92.5990.3147
ELM95.2694.7250
SaE-ELM95.8195.2740
SVM94.7594.1697a
SAHRBF-BP96.8296.5826&4b
B30GermanSGBP22.9239.168
MRAN68.4566.29137
ELM75.5470.50120
SaE-ELM77.5172.6380
SVM77.3876.39234a
SAHRBF-BP82.6281.54262 &9b
B31IonosphereSGBP73.3056.868
MRAN84.3482.2986
ELM89.7785.93100
SaE-ELM90.7588.1460
SVM91.2691.8250a
SAHRBF-BP93.5294.0454&6b
B38RetinopathySGBP41.4953.827
MRAN70.5472.28152
ELM73.1672.75100
SaE-ELM74.6274.5650
SVM74.3176.87339a
SAHRBF-BP77.2578.51107&7b
B42SpambaseSGBP57.0260.329
MRAN85.2886.56421
ELM90.3291.54100
SaE-ELM91.8291.6070
SVM91.6791.21862a
SAHRBF-BP93.2192.79284&9b
B46VoteSGBP73.2675.826
MRAN90.1589.2126
ELM93.7293.2740
SaE-ELM94.3694.1320
SVM93.5894.2831a
SAHRBF-BP95.1295.3415&4b

aSupport vectors.

bRBF&BP hidden nodes.

aSupport vectors. bRBF&BP hidden nodes. aSupport vectors. bRBF&BP hidden nodes. aSupport vectors. bRBF&BP hidden nodes. For Diabetes data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.7%, and SaE-ELM, ELM, MRAN by approximately 2.2%-7%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 5.2%, and SaE-ELM, ELM, MRAN by approximately 2.9%-7.9%. For Heart disease data set, the overall and average testing accuracies of SAHRBF-BP outperforms SaE-ELM, ELM, MRAN by approximately 0.6%-5.7%. The average testing accuracy of SAHRBF-BP is approximately 2.2% lower than that of SVM, however, the overall testing accuracy is higher than that of SVM by approximately 1.4%. For Mammographic data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.9%, and SaE-ELM, ELM, MRAN by approximately 3.2%-7%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.5%, and SaE-ELM, ELM, MRAN by approximately 3.3%-8.5%. For Monk1 data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.1%, and SaE-ELM, ELM, MRAN by approximately 6.1%-10.1%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.6%, and SaE-ELM, ELM, MRAN by approximately 6.5%-10.4%. For Monk2 data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.3%, and SaE-ELM, ELM, MRAN by approximately 9%-11.3%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.5%, and SaE-ELM, ELM, MRAN by approximately 9.1%-13%. For Svmguide1 data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.6%, and SaE-ELM, ELM, MRAN by approximately 2.1%-6%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.5%, and SaE-ELM, ELM, MRAN by approximately 1.4%-7.1%. For Vertebral data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.3%, and SaE-ELM, ELM, MRAN by approximately 2.7%-6%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.4%, and SaE-ELM, ELM, MRAN by approximately 2%-6.7%. For Wholesale data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.3%, and SaE-ELM, ELM, MRAN by approximately 1.8%-4.6%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.1%, and SaE-ELM, ELM, MRAN by approximately 2.1%-6.3%. For Climate data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.1%, and SaE-ELM, ELM, MRAN by approximately 1%-5.2%. The average testing accuracy of SAHRBF-BP is comparable with SVM, and outperforms SaE-ELM, ELM, MRAN by approximately 0.5%-5.7%. For Congressional data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.1%, and SaE-ELM, ELM, MRAN by approximately 1%-4.2%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.4%, and SaE-ELM, ELM, MRAN by approximately 1.3%-6.3%. For German data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 5.2%, and SaE-ELM, ELM, MRAN by approximately 5.1%-14.2%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 5.1%, and SaE-ELM, ELM, MRAN by approximately 8.9%-15.2%. For Ionosphere data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.2%, and SaE-ELM, ELM, MRAN by approximately 2.8%-9.2%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.2%, and SaE-ELM, ELM, MRAN by approximately 5.9%-11.7%. For Retinopathy data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.9%, and SaE-ELM, ELM, MRAN by approximately 2.6%-6.7%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.6%, and SaE-ELM, ELM, MRAN by approximately 4%-6.2%. For Spambase data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.5%, and SaE-ELM, ELM, MRAN by approximately 1.4%-8%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.6%, and SaE-ELM, ELM, MRAN by approximately 1.2%-6.2%. For Vote data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.5%, and SaE-ELM, ELM, MRAN by approximately 0.8%-5%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1%, and SaE-ELM, ELM, MRAN by approximately 1.2%-6.1%. In Table 6, for Fertility data set, the overall testing accuracy of SAHRBF-BP is lower than SGBP about 9%, lower than SVM about about 3.5%, SaE-ELM about 2.5%, and ELM about 2%. The average testing accuracy of SAHRBF-BP is lower than SGBP about 8.1%, and SVM, SaE-ELM, ELM about 4.5%, 3.4%, 2.8%, respectively. For Planning data set, the overall and average testing accuracies are lower than SGBP and SVM about 2.6%-3.2%. For Hill data set, the overall testing accuracy is lower than ELM, SVM and SaE-ELM about 2.3%-2.9%, and the average testing accuracy of SAHRBF-BP is lower than ELM, SVM, SaE-ELM about 1.7%-3.1%. For LSVT data set, the overall testing accuracy is lower than ELM about 0.6%, SaE-ELM about 2%, and SVM about 13.3%. The average testing accuracy of SAHRBF-BP is lower than ELM, SaE-ELM and SVM about 0.7%, 1%, 14.8%, respectively. For Musk1 data set, the overall and average testing accuracies are lower than SVM about 8.8% and 9.4%, respectively. For QSAR data set, the overall and average testing accuracies are lower than SVM about 1.4% and 1.7%, respectively. For Sonar data set, the overall testing accuracy of SAHRBF-BP is lower than SVM about 3.1%, and the average testing accuracy of SAHRBF-BP is lower than SVM about 8.4%. For Spect heart data set, the overall and average testing accuracies are lower than SGBP clearly. The overall testing accuracy of SAHRBF-BP is lower than SaE-ELM, SVM about 0.5%, 2.5%, respectively. The average testing accuracy of SAHRBF-BP is lower than SaE-ELM and SVM about 2%, 6.7%, respectively. From Fig 13 and Tables 4–6 we can see that for most binary class low dimensional data sets, the classification accuracy of SAHRBF-BP is higher than other learning algorithms. However, for binary class high dimensional data sets, the classification accuracy of SAHRBF-BP is decreased clearly on a number of data sets, such as LSVT(B32), Musk1(B34), QSAR(B37), Secom(B39), Sonar(B41) data sets. The main reasons are that, with the increase of dimension, spatial distribution of the samples is relatively sparse, especially for small number of training samples data sets, the randomness of training samples in the sample space is greatly enhanced, which can not effectively reflect the actual distribution of entire data sets, and leads to a certain degree of failure by the methods of potential function clustering and heterogeneous samples repulsive force. Thus, the classification performance of SAHRBF-BP will be reduced to varying degrees.

Benchmark multi-class classification problems

In this section, 18 multi-class low dimensional data sets and 17 multi-class high dimensional data sets are used to evaluate the performance of SAHRBF-BP. Fig 14 shows the overall testing accuracy comparisons between SAHRBF-BP and other learning algorithms. For multi-class low dimensional data sets, the overall testing accuracy of SAHRBF-BP is comparable with SVM on Teaching(M13) data set, and is lower than SVM on Breast tissue(M02), Hayes-Roth(M08) data sets, lower than SaE-ELM, ELM on Glass(M07) data set, lower than SaE-ELM, SVM, ELM on Iris(M09) data set, however, the overall testing accuracy of SAHRBF-BP is higher than other learning algorithms on the rest 13 data sets.
Fig 14

Overall testing accuracy comparisons between SAHRBF-BP and other algorithms on benchmark multi-class data sets.

(A) Multi-class low dimensional data sets (B) Multi-class high dimensional data sets.

Overall testing accuracy comparisons between SAHRBF-BP and other algorithms on benchmark multi-class data sets.

(A) Multi-class low dimensional data sets (B) Multi-class high dimensional data sets. For multi-class high dimensional data sets, the overall testing accuracy of SAHRBF-BP is higher than other learning algorithms on Firm(M22), Image segmentation(M25), Landsat(M26), Steel(M30), Turkiye(M31), Vehicle silhouettes(M32), Waveform1(M33) and Waveform2(M34) data sets. The overall testing accuracy of SAHRBF-BP is comparable with SVM on Gas(2012)(M24) data set, and SVM, ELM, SaE-ELM on Optical digits(M28) and Semeion(M29) data sets, however, the overall testing accuracy of SAHRBF-BP is lower than SVM on Air(M19), DNA(M21), Forest(M23) and Libras(M27) data sets, and ELM, SaE-ELM, SVM on Dermatology(M20) and Zoo(M35) data sets. Tables 7–9 give performance comparisons between SAHRBF-BP and other learning algorithms on partial multi-class data sets, where the overall and average testing accuracies of SAHRBF-BP are clearly higher than SGBP. In Tables 7 and 8, for Balance data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.5%, and SaE-ELM, ELM, MRAN by approximately 1%-6%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.3%, and SaE-ELM, ELM, MRAN by approximately 1.5%-7.5%.
Table 7

A few cases of success in SAHRBF-BP compared with other learning algorithms on benchmark multi-class data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
M01BalanceSGBP71.2368.517
MRAN85.6183.2947
ELM88.7286.6450
SaE-ELM90.6489.3340
SVM90.1389.5265a
SAHRBF-BP91.6790.8529&4b
M04CardiotocographySGBP42.5746.7311
MRAN51.7452.84387
ELM58.6656.51300
SaE-ELM59.5758.52200
SVM60.2161.751378a
SAHRBF-BP61.5262.58281&9b
M10KnowledgeSGBP68.5265.407
MRAN78.2576.1365
ELM79.0778.6150
SaE-ELM80.2479.2530
SVM80.6581.23184a
SAHRBF-BP83.1682.2456&5b
M12SeedsSGBP70.58–7
MRAN62.42–26
ELM64.76–30
SaE-ELM65.29–20
SVM72.47–47a
SAHRBF-BP74.12–13&4b
M15WineSGBP86.13–8
MRAN89.16–46
ELM96.12–30
SaE-ELM97.18–22
SVM96.36–32a
SAHRBF-BP97.54–16&5b
M18YeastSGBP42.2640.539
MRAN55.5254.72323
ELM56.4155.65200
SaE-ELM57.5356.33120
SVM59.2758.34852a
SAHRBF-BP61.8460.82216&8b
M25Image segmentationSGBP90.16–8
MRAN85.49–81
ELM90.31–100
SaE-ELM91.17–50
SVM90.56–96a
SAHRBF-BP92.23–32&6b
M26LandsatSGBP74.8378.8412
MRAN87.2685.42462
ELM90.3490.13600
SaE-ELM90.8290.38350
SVM91.2190.871618a
SAHRBF-BP91.9491.82312&9b

aSupport vectors.

bRBF&BP hidden nodes.

Table 9

A few cases of failures in SAHRBF-BP compared with other learning algorithms on benchmark multi-class data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
M02Breast tissueSGBP54.2153.496
MRAN54.5153.2512
ELM57.6057.2830
SaE-ELM58.3357.7620
SVM59.5258.8431a
SAHRBF-BP58.2657.4215&6b
M07GlassSGBP58.4357.877
MRAN63.7962.6131
ELM66.7266.2440
SaE-ELM67.2666.8220
SVM65.1464.8394a
SAHRBF-BP64.8463.2013&7b
M08Hayes-RothSGBP68.7366.537
MRAN70.7669.3227
ELM71.4370.2740
SaE-ELM72.2671.8030
SVM74.1274.5868a
SAHRBF-BP72.6871.4212&5b
M19AirSGBP72.1474.638
MRAN85.5984.5142
ELM86.2685.6450
SaE-ELM87.0387.2330
SVM88.4288.1692a
SAHRBF-BP87.2786.4538&6b
M23ForestSGBP45.2644.528
MRAN66.2564.7173
ELM68.5667.6360
SaE-ELM68.8468.3840
SVM72.1974.53125a
SAHRBF-BP69.9369.3751&5b
M27LibrasSGBP42.69–9
MRAN50.53–96
ELM52.78–120
SaE-ELM52.26–80
SVM56.24–243a
SAHRBF-BP54.67–82&7b

aSupport vectors.

bRBF&BP hidden nodes.

Table 8

A few cases of success in SAHRBF-BP compared with other learning algorithms on benchmark multi-class data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
M30SteelSGBP68.5766.799
MRAN64.5662.82372
ELM72.3871.53200
SaE-ELM72.8571.61130
SVM71.5170.87263a
SAHRBF-BP73.7272.86274&9b
M31TurkieSGBP45.8442.1214
MRAN53.2552.87534
ELM61.5060.64200
SaE-ELM62.2761.89100
SVM62.8262.583237a
SAHRBF-BP64.2663.80415&10b
M32Vehicle silhouettesSGBP72.6473.829
MRAN59.3659.78102
ELM76.8477.08150
SaE-ELM77.7378.23120
SVM73.7969.74247a
SAHRBF-BP78.2178.6593&7b
M33Waveform1SGBP53.2660.3911
MRAN83.3682.28371
ELM85.9284.86200
SaE-ELM86.7186.24100
SVM86.3786.221854a
SAHRBF-BP87.5387.13326&9b

aSupport vectors.

bRBF&BP hidden nodes.

aSupport vectors. bRBF&BP hidden nodes. aSupport vectors. bRBF&BP hidden nodes. aSupport vectors. bRBF&BP hidden nodes. For Cardiotocography data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.3%, and SaE-ELM, ELM, MRAN by approximately 2%-9.8%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.8%, and SaE-ELM, ELM, MRAN by approximately 4%-9.7%. For Knowledge data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.5%, and SaE-ELM, ELM, MRAN by approximately 2.9%-4.9%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1%, and SaE-ELM, ELM, MRAN by approximately 3%-6.1%. For Seeds data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.6%, and SaE-ELM, ELM, MRAN by approximately 8.8%-11.7%. For Wine data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.2%, and SaE-ELM, ELM, MRAN by approximately 0.4%-8.4%. For Yeast data set, the overall and average testing accuracies of SAHRBF-BP outperform SVM by approximately 2.5%, and SaE-ELM, ELM, MRAN by approximately 4.3%-6.3%. For Image segmentation data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.7%, and SaE-ELM, ELM, MRAN by approximately 1.1%-6.7%. For Landsat data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.7%, and SaE-ELM, ELM, MRAN by approximately 1.1%-4.7%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1%, and SaE-ELM, ELM, MRAN by approximately 1.4%-6.4%. For Steel data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 2.2%, and SaE-ELM, ELM, MRAN by approximately 0.9%-9.1%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 2%, and SaE-ELM, ELM, MRAN by approximately 1.2%-10%. For Turkie data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.4%, and SaE-ELM, ELM, MRAN by approximately 2%-11%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.4%, and SaE-ELM, ELM, MRAN by approximately 1.9%-10.9%. For Vehicle silhouettes data set, the overall and average testing accuracies of SAHRBF-BP are clearly higher than MRAN. The overall testing accuracy outperforms SVM by approximately 4.4%, and SaE-ELM, ELM by approximately 0.5%-1.6%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 8.9%, and SaE-ELM, ELM by approximately 0.4%-1.5%. For Waveform1 data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.2%, and SaE-ELM, ELM, MRAN by approximately 0.8%-4.2%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.9%, and SaE-ELM, ELM, MRAN by approximately 0.9%-4.9%. In Table 9, for Breast tissue data set, the overall and average testing accuracies of SAHRBF-BP are lower than SVM about 1.3% and 1.4%, respectively. For Glass data set, the overall testing accuracy of SAHRBF-BP is lower than ELM about 1.9%, SaE-ELM about 2.4%, and SVM about 0.3%. The average testing accuracy of SAHRBF-BP is lower than ELM, SaE-ELM and SVM about 3%, 3.6%, 1.6%, respectively. For Hayes-Roth data set, the overall and average testing accuracies of SAHRBF-BP are lower than SVM about 1.4% and 3.2%, respectively. For Air data set, the overall testing accuracy of SAHRBF-BP is lower than SVM about 1.2%. The average testing accuracy of SAHRBF-BP is lower than SVM about 1.7%, and SaE-ELM about 0.8%. For Forest data set, the overall and average testing accuracies of SAHRBF-BP are lower than SVM about 2.3% and 5.2%, respectively. For Libras data set, the overall testing accuracy of SAHRBF-BP is lower than SVM about 1.6%. Similar to benchmark binary class data sets, the overall classification accuracy of SAHRBF-BP is higher than other learning algorithms on most multi-class low dimensional data sets, and the overall classification accuracy of SAHRBF-BP is decreased on a number of multi-class high dimensional data sets. However, for multi-class high dimensional data sets, when the number of training samples is sufficient, a relatively high classification accuracy of SAHRBF-BP can be obtained. The main reason is that enough training samples can offset the random distribution of sample space to a great extent. Under these circumstances, the methods of potential function clustering and heterogeneous samples repulsive force are still valid.

Benchmark large number of samples classification problems

In this section, 27 large number of samples data sets are used to evaluate the performance of SAHRBF-BP. Fig 15 shows the overall testing accuracy comparisons between SAHRBF-BP and other learning algorithms. For large number of samples data sets, the overall testing accuracy of SAHRBF-BP is comparable with other learning algorithms on A1a(adult)(L01), A6a(adult)(L02) data sets, and SVM, SaE-ELM, ELM on Cod_rna(L15), Credit(L16), Gas(2013)(L18), Record(L23), Sensorless(L24), Skin(L25), Shuttle(L26) data sets. The overall testing accuracy of SAHRBF-BP is slightly lower than SaE-ELM, ELM on Letter(L20) data set. For other data sets, the overall testing accuracy of SAHRBF-BP is higher than other learning algorithms to varying degrees.
Fig 15

Overall testing accuracy comparisons between SAHRBF-BP and other algorithms on large number of samples data sets.

Table 10 gives performance comparisons between SAHRBF-BP and other learning algorithms on partial large number of samples data sets. The overall and average testing accuracies of SAHRBF-BP are clearly higher than SGBP on each large number of samples data set, except for A1a(adult) and A6a(adult) data sets. For Action2(normal)(L04) data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.7%, and SaE-ELM, ELM, MRAN by approximately 1.1%-9.6%. For Action1(aggressive) data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.8%, and SaE-ELM, ELM, MRAN by approximately 1.7%-2.1%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.2%, and SaE-ELM, ELM, MRAN by approximately 1.8%-2.5%. For Action2(abnormal detection) data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.2%, and SaE-ELM, ELM, MRAN by approximately 1%-8.7%. For Action3(abnormal detection) data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.4%, and SaE-ELM, ELM, MRAN by approximately 1.2%-8.2%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.9%, and SaE-ELM, ELM, MRAN by approximately 1%-8.2%. For Ijcnn1 data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 0.7%, and SaE-ELM, ELM, MRAN by approximately 1.3%-11%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.1%, and SaE-ELM, ELM, MRAN by approximately 2%-11.5%. For letter data set, the overall and average testing accuracies of SAHRBF-BP outperform SVM by approximately 0.5%, and MRAN by approximately 9.6% and 9.8%, respectively. However, the overall and average testing accuracy of SAHRBF-BP is lower than SaE-ELM about 0.4%, and ELM about 0.6% and 0.5%, respectively. For Occupancy data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.2%, and SaE-ELM, ELM, MRAN by approximately 1%-8.7%. For Action3(abnormal detection) data set, the overall testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.4%, and SaE-ELM, ELM, MRAN by approximately 2.2%-8.5%. The average testing accuracy of SAHRBF-BP outperforms SVM by approximately 1.5%, and SaE-ELM, ELM, MRAN by approximately 2.7%-8.6%.
Table 10

Performance comparisons between SAHRBF-BP and other learning algorithms on partial large number of samples data sets.

IndexData setsMethodsTesting ηoTesting ηaNH nodes
L01A1a(adult)SGBP1001008
MRAN10010076
ELM100100100
SaE-ELM10010050
SVM100100386a
SAHRBF-BP10010058&5b
L04Action2(normal)SGBP38.52–40
MRAN43.29–2441
ELM51.36–4000
SaE-ELM51.74–3500
SVM52.12–5838a
SAHRBF-BP52.87–2123&20b
L07Action1(aggressive)SGBP37.0636.6440
MRAN42.4842.212549
ELM42.5842.363500
SaE-ELM42.8642.913000
SVM43.7743.546256a
SAHRBF-BP44.6144.732258&22b
L12Action2(abnormal detection)SGBP28.17–40
MRAN75.12–3225
ELM81.72–4000
SaE-ELM82.83–3200
SVM82.56–5515a
SAHRBF-BP83.81–2432&24b
L13Action3(abnormal detection)SGBP27.2128.6240
MRAN75.6775.393173
ELM81.9881.824000
SaE-ELM82.7282.683200
SVM82.5482.765584a
SAHRBF-BP83.9283.642471&24b
L19Ijcnn1SGBP35.2847.6240
MRAN83.5782.251832
ELM92.9390.823000
SaE-ELM93.2691.782000
SVM93.8192.613974a
SAHRBF-BP94.5693.741657&18b
L20LetterSGBP62.7262.2430
MRAN82.6582.312630
ELM92.8392.673000
SaE-ELM92.6192.572000
SVM91.7491.583668a
SAHRBF-BP92.2692.122146&20b
L21OccupancySGBP42.2640.8330
MRAN72.1471.621263
ELM76.5275.181400
SaE-ELM78.3077.381000
SVM79.2478.661863a
SAHRBF-BP80.6780.21862&13b

aSupport vectors.

bRBF&BP hidden nodes.

aSupport vectors. bRBF&BP hidden nodes. From Fig 15 and Table 10, we can see that for large number of samples data sets, the classification accuracy of SAHRBF-BP is higher than other learning algorithms in general. Enough training samples can effectively reflect the actual distribution of entire data sets, and the superiority of potential function clustering and heterogeneous samples repulsive force can be fully demonstrated.

Discussion

Selection of the initial width parameters for SAHRBF-BP

The width parameters can be used to control the classification accuracy and generalization performance. To optimize the coverage of each class of samples, the center adjustment and the width adjustment strategy are combined together. When an initial width is given, for each generated RBF hidden node, the center is iteratively adjusted to a suitable position, and the width is then adjusted only once. To reduce the range of initial width values, we execute a preprocessing step for the sample space. For all benchmark classification problems, the inputs to each algorithm are scaled appropriately to fall between -1 and +1. In addition, the initial width σ and the minimum width σmin are related to each other. According to Eq (16), the adjusted width is in the range between σmin and σ; that is, , where σmin ∈ {σmin|σ − ϑ ≤ σmin < σ, σmin > 0}. To guarantee the generalization performance, here we set ϑ = 0.2. Thus, when the initial width σ is given, the minimum width σmin can be determined accordingly. For example, if σ = 0.5, σmin can be selected in the set {σmin|σ − 0.2 ≤ σmin < σ}. To simplify this case, σmin can be selected in the set {0.3, 0.4}, and the one with the lowest validation error is selected as the suitable minimum width.

Effect of the initial width parameters on SAHRBF-BP

When the initial width changes, the number of generated RBF hidden nodes and node parameters will change accordingly. Here Diabetes, Heart disease, Ionosphere, Image segmentation and Vehicle silhouettes data sets are used to evaluate the effect of the initial width parameters on SAHRBF-BP. In Fig 16, when the initial width value is too small, the overall classification accuracy is poor and the network size for the RBF hidden layer is large, e.g., for Heart disease data set, when σ = 0.2, the number of generated RBF hidden neurons is 151 and is equal to the number of training samples.
Fig 16

Effect of initial width parameters on SAHRBF-BP.

(A) σ- Overall testing accuracy (B) σ- Number of RBF hidden nodes.

Effect of initial width parameters on SAHRBF-BP.

(A) σ- Overall testing accuracy (B) σ- Number of RBF hidden nodes. This result demonstrates the corresponding RBF hidden nodes will be established at each training sample, and the generated RBF hidden neurons will not cover other samples, thus the methods of potential function clustering and heterogeneous sample repulsive force are invalid and the overall classifying accuracy is poor in this case. Thus, in SAHRBF-BP, to complete the effective coverage of the training sample space, an effective initial width parameter should be provided, which can generate proper RBF hidden neurons to cover the sample space. Note that the number of generated RBF hidden neurons should not be close to the number of the training samples, otherwise, SAHRBF-BP is invalid. When the value of the initial width falls within a suitable range, the number of generated RBF hidden nodes will change, but a relatively stable classification accuracy can be achieved. For instance, for Image segmentation data set, when the range of initial widths is between 0.5 and 0.9, the overall testing accuracy ranges from 91.54% to 92.23%. Once initial width parameters are given, SAHRBF-BP can learn the sample space automatically and generate different RBF hidden nodes to adapt the sample space. Thus, SAHRBF-BP can counteract the effect of the initial width parameters to some extent.

Effect of the number of BP hidden nodes on SAHRBF-BP

In SAHRBF-BP, the nonlinear SGBP algorithm is used to adjust the weights of the BP network component, which further improves the classification result. However, this method results in an increase in the number of parameters to be selected, particularly the selection of the number of BP hidden nodes. For this problem, we conduct experiments on five UCI data sets and discuss the results. Fig 17 shows the effect of the number of BP hidden nodes on SAHRBF-BP. The results show that for binary class classification problems, such as Diabetes, Heart disease and Ionosphere data sets, when the number of BP hidden nodes ranges from 1 to 10, a relatively stable classification accuracy can be achieved.
Fig 17

Effect of the number of BP hidden nodes on SAHRBF-BP.

For multi-class classification problems, such as Image segmentation and Vehicle silhouettes data sets, when the number of BP hidden nodes is greater than 4, the overall classification accuracy also does not change considerably. Thus, the dependence on the number of BP hidden nodes is reduced. For the SAHRBF-BP classifier, the adaptively mapping results for the RBF hidden nodes are processed and used for the input of BP network component, which improves the stability of the BP network component and effectively avoids falling into local minima for the BP algorithm. When the sample set is more complex, the momentum term can be used to improve the BP algorithm further.

Limitations for SAHRBF-BP

Compared with other training SLFNs algorithms, SAHRBF-BP shows excellent classification performance on artificial and most benchmark data sets. However, there are still some limitations for SAHRBF-BP. For complex classification problems, to achieve good classification results, the number of training samples should not be too small. Otherwise, the randomness of training samples in the sample space is enhanced, which can not effectively reflect the actual distribution of entire data sets, especially for high dimensional data sets, and will lead to the methods of potential function clustering and heterogeneous sample repulsive force some extent of failure. In addition, to ensure the effectiveness of learning, the initial kernel width should not be too small, which is another limitation for SAHRBF-BP. Otherwise, the generalization performance of the classifier will be greatly reduced, and each generated RBF hidden node does not cover the heterogeneous samples, which leads to the failure of heterogeneous sample repulsive force.

Conclusion

In this paper, a structure-adaptive hybrid RBF-BP (SAHRBF-BP) classifier with an optimized learning strategy is presented. SAHRBF-BP is composed of a structure-adaptive RBF network and a BP network of cascade, where the number of RBF hidden nodes is adjusted adaptively according to the distribution of sample space. SAHRBF-BP makes use the global information of each class of training samples to generate the initial RBF hidden nodes, and then makes full use of the neighborhood information of each hidden node to optimize the hidden node parameters. Thus, SAHRBF-BP solves the problem of dimension change from sample space mapping onto feature space. In addition, it also effectively combines the stability of a RBF network and the generalization ability of a BP network to improve the classification performance. In this way, SAHRBF-BP simplifies the selection of the number of nodes in the BP hidden layer while further reducing the dependence on space mapping in the RBF hidden layer. The optimized learning strategy can generate RBF hidden nodes incrementally, as well as adjust the centers and width adaptively. The combination of the potential function clustering with heterogeneous sample repulsive force improves the classification accuracy of each hidden node; at the same time, it ensures a compact network size for the RBF hidden layer. The performance of SAHRBF-BP is compared with that of other training SLFNs algorithms, namely SGBP, KMRBF, KMRBF-BP, MRAN, GAP-RBF, SVM, ELM, and SaE-ELM on different data sets. In each training SLFNs algorithm, SVM is still the most stable classifier. Compared to other algorithms, the classification performance of SVM is maintained at a relatively high level on each data set in general. Overall, for high dimensional with too small training samples data sets, the classification performance of SVM outperforms SAHRBF-BP clearly. However, with the increase of the number of samples in data sets, the randomness of training samples in the sample space is gradually being eliminated. On the basis of effective learning of sample space, SAHRBF-BP shows its unique advantages. On most low dimensional and large number of data sets, the results show the classification performance of SAHRBF-BP outperforms other training SLFNs algorithms. In the future, we will focus on imbalanced data classification problems. For imbalanced data classification problems, the samples of minority classes and the samples in boundary regions should be emphasized more, which contain more classification information, thus how to measure and select these samples is particularly important. Further studies are needed to address these concerns.
Table 2

Descriptions of multi-class data sets.

IndexData sets# Classes# Features# Training# Validation# Testing
M01Balance34300150175
M02Breast tissue69532726
M03Car46864432432
M04Cardiotocography1091063532531
M05Contraceptive39736369368
M06Ecoli871688484
M07Glass691095352
M08Hayes-Roth351003228
M09Iris34753639
M10Knowledge45130128145
M11Page510273713681368
M12Seeds371055352
M13Teaching35753838
M14Vowel1110264132132
M15Wine313904345
M16Wine quality(red)612800400399
M17Wine quality(white)712245010003448
M18Yeast810742371371
M19Air36417881100
M20Dermatology6341839291
M21DNA31802000593593
M22Firm416540027002700
M23Forest427198100225
M24Gas(2012)6128291910001000
M25Image segmentation7192104201680
M26Landsat636443510001000
M27Libras15903003030
M28Optical digits10643823900897
M29Semeion10256796400397
M30Steel720971385385
M31Turkiye332291014551455
M32Vehicle silhouettes418424211211
M33Waveform1321250012501250
M34Waveform2340250012501250
M35Zoo16101512525
  14 in total

1.  Metacognitive learning in a fully complex-valued radial basis function neural network.

Authors:  R Savitha; S Suresh; N Sundararajan
Journal:  Neural Comput       Date:  2011-12-14       Impact factor: 2.026

2.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks.

Authors:  Guang-Bin Huang; P Saratchandran; Narasimhan Sundararajan
Journal:  IEEE Trans Syst Man Cybern B Cybern       Date:  2004-12

3.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation.

Authors:  Guang-Bin Huang; P Saratchandran; Narasimhan Sundararajan
Journal:  IEEE Trans Neural Netw       Date:  2005-01

4.  An incremental training method for the probabilistic RBF network.

Authors:  Constantinos Constantinopoulos; Aristidis Likas
Journal:  IEEE Trans Neural Netw       Date:  2006-07

5.  Universal approximation using incremental constructive feedforward networks with random hidden nodes.

Authors:  Guang-Bin Huang; Lei Chen; Chee-Kheong Siew
Journal:  IEEE Trans Neural Netw       Date:  2006-07

6.  Conditional fuzzy clustering in the design of radial basis function neural networks.

Authors:  W Pedrycz
Journal:  IEEE Trans Neural Netw       Date:  1998

7.  Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques.

Authors:  N B Karayiannis; G W Mi
Journal:  IEEE Trans Neural Netw       Date:  1997

8.  Error minimized extreme learning machine with growth of hidden nodes and incremental learning.

Authors:  Guorui Feng; Guang-Bin Huang; Qingping Lin; Robert Gay
Journal:  IEEE Trans Neural Netw       Date:  2009-07-10

9.  A sequential learning scheme for function approximation using minimal radial basis function neural networks.

Authors:  Y Lu; N Sundararajan; P Saratchandran
Journal:  Neural Comput       Date:  1997-02-15       Impact factor: 2.026

10.  An incremental design of radial basis function networks.

Authors:  Hao Yu; Philip D Reiner; Tiantian Xie; Tomasz Bartczak; Bogdan M Wilamowski
Journal:  IEEE Trans Neural Netw Learn Syst       Date:  2014-02-11       Impact factor: 10.451
View more
  2 in total

1.  The Lords of the Rings: People and pigeons take different paths mastering the concentric-rings categorization task.

Authors:  Ellen M O'Donoghue; Matthew B Broschard; John H Freeman; Edward A Wasserman
Journal:  Cognition       Date:  2021-10-04

2.  Prediction of Mumps Incidence Trend in China Based on Difference Grey Model and Artificial Neural Network Learning.

Authors:  Jin Jia; Mingming Liu; Zhigang Xue; Zhe Wang; Yu Pan
Journal:  Iran J Public Health       Date:  2021-07       Impact factor: 1.429
  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.