Medical Dataset Classification: A Machine Learning Paradigm Integrating Particle Swarm Optimization with Extreme Learning Machine Classifier.

Medical data classification is a prime data mining problem being discussed about for a decade that has attracted several researchers around the world. Most classifiers are designed so as to learn from the data itself using a training process, because complete expert knowledge to determine classifier parameters is impracticable. This paper proposes a hybrid methodology based on machine learning paradigm. This paradigm integrates the successful exploration mechanism called self-regulated learning capability of the particle swarm optimization (PSO) algorithm with the extreme learning machine (ELM) classifier. As a recent off-line learning method, ELM is a single-hidden layer feedforward neural network (FFNN), proved to be an excellent classifier with large number of hidden layer neurons. In this research, PSO is used to determine the optimum set of parameters for the ELM, thus reducing the number of hidden layer neurons, and it further improves the network generalization performance. The proposed method is experimented on five benchmarked datasets of the UCI Machine Learning Repository for handling medical dataset classification. Simulation results show that the proposed approach is able to achieve good generalization performance, compared to the results of other classifiers.

1. Introduction

In recent times, the application of computational or machine intelligence in medical diagnosis is a new trend for large medical data applications. Most of the diagnosis techniques in medical field are systematized as intelligent data classification approaches. In computer-aided decision (CAD) systems, information technology methods are adopted to assist a physician to diagnose disease of a patient. Among the various assignments performed by a CAD system, classification is most common, where a tag is allocated to a query case (i.e., a patient) based on chosen number of features (i.e., medical findings). Thus medical database classification problem may be categorized as a class of complex optimization problem with an objective to guarantee the diagnosis aid accurately. Aside from other traditional classification problems, medical dataset classification problems are also applied in future diagnosis. Generally, patients or doctors are not completely informed about the cause (classification result) of the disease, but also will be made known of the symptoms that derive the cause of disease, which is the most important of the medical dataset classification problem.

In a classification problem, the objective is to learn the decision surface that accurately maps an input feature space to an output space of class labels [1]. In medical field, various computer researchers have attempted to apply diverse techniques to improve the accuracy of data classification for the given data, classification techniques whose classification accuracy will better yield enough information to identify the potential patients and thereby improvise the diagnosis accuracy. In the recent studies, metaheuristic algorithms (to name a few, simulated annealing, genetic algorithms, and particle swarm optimizations) and also data mining techniques (to name a few, Bayesian networks, artificial neural network, fuzzy logic, and decision tree) were applied for classification of medical data and obtained with remarkably meaningful results.

As per literature and above discussion, for past decades several classification tools are available for medical dataset classification. Even then, artificial neural networks (ANNs) are widely accepted and utilized to solve the real world classification problems in clinical applications. Because of their generalization and conditioning capabilities, requirement of minimal training points, and faster convergence time, ANNs are found to perform better and result in faster output in comparison with the conventional classifiers [2]. Various learning/training algorithms for several neural network architectures have been proposed in various problems in science, engineering, and technology and even in some parts of business industry and medicine. A few notable classification applications include the handwritten character recognition, speech recognition, biomedical medical diagnosis, text categorization, information retrieval, and prediction of bankruptcy [3, 4].

Several versions of ANN are modeled, to name a few, feedforward neural network, Boltzmann machine, radial basis function (RBF) network, Kohonen self-organizing network, learning vector quantization, recurrent neural network, Hopfield network, spiking neural networks, and extreme learning machine (ELM); most of them are inspired by biological neural networks. Among these several ANNs, feedforward neural networks (FFNNs) are popularly and widely used for the classification/identification of linear/nonlinear systems. To prevail over the slow construction of FFNN models, several new training schemes were introduced and amongst them the extreme learning machine (ELM) has gained wide attention in recent days [4]. One of the very unusual individualities of this ELM is its nontraditional training procedure. Here, ELM randomly selects the input layer and hidden layer connection weights and also estimates the connection weights between the hidden layer and output layer analytically. Besides, the ELM tends to require more hidden neurons compared to conventional tuning-based learning algorithms; perhaps this has been believed to be trivial compared to other positive features of ELM [5].

In spite of its superiority with other FFNN training algorithms, various advancements have been proposed to improvise its performance in the last few years, merely by hybridizing the ELM with recent metaheuristic algorithms. Hybridization was done with two ideas: one is a feature selection approach using the ELM as wrapper classifier [6] and the other theme is using microevolution in order to obtain the best set of weights and biases in the input layer of the ELM [7]. This research opts for the second category, where one of the very effective metaheuristic algorithms has been used jointly with analytical methods for FFNN training. With a growing body of literature in this category of hybridization for training the ELM and applications of classifiers for medical set classification, a comprehensive list of a few such literatures is given in [7-26].

In [27] a new nonlinear system identification scheme is proposed, where differential evolution (DE) is used to optimize the initial weights used by a Levenberg-Marquardt (LM) algorithm in the learning of a FFNN. In [28] a similar method was proposed using a simulated annealing (SA) approach and subsequently the training of FFNN with a back propagation method, which is computationally expensive. In [29], the process of selecting the input weights and hidden biases, using an improved PSO and validation set of the output weights as well as constrained input weights and hidden biases, was within a reasonable range. Again in [30] the coral reefs optimization (CRO) has been used for fast convergence to optimal values and had been used for carrying out evolution in ELM weights, in order to enhance the performance of these machines.

In this paper, again a PSO based hybrid algorithm to train the ELM is proposed. The proposed hybrid algorithm is modeled in such a way that the ELM solves the main problem, whereas the PSO evolves the weights of the FFFN, to improve the solutions obtained and further to improve the network generalization performance. In order to obtain a robust SLFN, otherwise the ELM for classification accuracy, the PSO algorithm is enhanced by incorporating a velocity updating mechanism which diversifies the search process of the particles and could escape all local traps and guarantees much better solutions for the ELM. The proposed hybrid model is able to classify some of the UCI medical datasets, namely, Wisconsin Breast Cancer, Pima Indians Diabetes, Heart-Statlog, Hepatitis, and Cleveland Heart Disease. The medical data classification was carried out by extracting and analyzing available data with a suitable sampling procedure.

This paper is organized as follows. The FFFN architecture for ELM is overviewed in Section 2. Section 3 gives a brief review of the ELM. The proposed improved particle swarm optimization (PSO) algorithm is presented in Section 4. Section 5 gives a detailed formulation of the proposed optimization methods for the ELM. Section 6 presents experimental results. Finally, concluding remarks are drawn in Section 7.

2. ELM for Medical Dataset Classification

This research for medical data classification relies on the performance of the extreme learning machine (ELM) classifier proposed in [4], which handles the training for single-hidden layer feedforward neural networks. An introduction to the ELM will be presented in the next section.

2.1. Extreme Learning Machine Classifier

The extreme learning machine (ELM) was originally developed in 1992 [3, 4] and can be categorized as a supervised learning algorithm capable of solving linear and nonlinear classification problems. When compared to other neural networks architectures, ELM may be understood as a single layer feedforward neural net (FFNN) with one hidden layer. The prime constituting blocks of ELMs are structural risk minimization, originating from statistical learning theory, nonlinear optimization, and duality and kernel induced features spaces, underlining the technique with an exact mathematical framework.

ELM [5, 6] is best suited for larger training samples and also the effect of number of hidden neurons using different ratios of the number of features of testing and training data was examined. This classifier is compared with that of the conventional neural network classifiers using the classification rate for medical data classification. The main motivation of the extreme learning machine classification is to separate classification data with a linear decision surface and maximize the margin in between the various categories of classes. This leads to the convex quadratic programming problem. It can be seen that training the ELM involves solving a quadratic optimization problem which requires the use of optimization routines from various recent mathematical or heuristic approaches. Specifically ELM classifier is chosen for considered application due to the following facts: extreme learning machine classifier provides good solutions, even for difficult search spaces and when complex datasets are used as in this case, extreme learning machine (ELM) is found to be a competitive good solution provider due to its converging characteristics. Further, extreme learning machine classifier reduces the computational burden and time prevailed in earlier classifiers. ELM achieves good generalization performance at extremely fast learning speed. Figure 1 shows the basic ELM architecture.

Figure 1

Architecture of the ELM (single layer FFNN).

The basic ELM classifier algorithm is given as follows.

Given N different samples (x , y ) ∈ R × R , where x = [x , x ,…,x ] is the input vector and y = [y , y ,…,y ] is the target vector, standard SLFNs with M hidden nodes and activation function g(x) are formulated aswhere w = [w , w ,…,w ] is the weight vector connecting the input nodes and the ith hidden nodes, α = [α , α ,…,α ] is the weight vector connecting the ith hidden node and the output nodes, and b is the bias of the ith hidden node. Assume that the function approaches all N samples by zero error; that is, there exist parameters (w , b ) and α such thatThe N equations can be simplified as α = T, whereThe solution of the linear system is α = H T, where H is the Moore-Penrose generalized inverse of hidden layer output matrix H. ELM algorithm can be written as the following three steps.

Input: it is a training dataset {(x , y )∣x ∈ ∣R , t ∈ R , j = 1,…, N}, an activation function g, and the number of hidden nodes M.

Output: it is the weights of hidden layer to output layer.

Randomly generate the parameters of weights and bias (w , b ), i = 1,…, M.

Calculate the hidden layer output matrix H.

Calculate the weights matrix of output α = H T.

Compared to general artificial neural networks the ELM method proffers a considerably smaller number of parameters for tuning. The main modeling idea consists in the choice of a kernel function and the equivalent kernel parameters have control over the convergence speed and the quality of final solution obtained.

3. Particle Swarm Optimization: An Overview

In information technology era swarm intelligence is the domain that derives its models from natural and artificial systems comprised of many individuals that coordinate using self-organization and decentralized control. One of the swarm intelligence methods, the particle swarm optimization (PSO) algorithm, was first proposed by Kennedy and Eberhart [32, 33]. It is inspired by observations from social dynamics of bird flocking or fish schooling. The idea arises from the natural behavior that a large number of birds flock parallelly, change direction spontaneously, scatter and regroup at intervals, and finally reach a target. This form of social behaviour increases the success rate for food foraging and expedites the process of reaching a target. This PSO algorithm simulating bird foraging or bee hiving activity can be modeled as an optimizer for nonlinear functions of continuous and discrete variables. Several literatures have already detailed the PSO; hence we just give a simple flowchart of the PSO as in Figure 2.

Figure 2

Flowchart of the basic PSO.

3.1. PSO Algorithm: Explained

The PSO algorithm searches the solution space using a swarm of n particles (i.e., [x 1, x 2, x 3,…, x ]), randomly positioned in the search space of the optimization problem. Each particle, which denotes a possible solution, is represented as a position and a velocity given by . For every algorithmic iteration, the ith particle position evolves using the following update rules: where w is the linear inertia constant, c 1, c 2 are the acceleration factors for cognitive and social component, respectively, r 1, , r 2, are random numbers generated in [0,1], p is the personnel best position of the ith particle, and p is the best among all the personnel bests in the entire particles in the current iteration k.

In order to improve the exploration and exploitation capacities of the proposed PSO algorithm, we choose for the inertia factor a linear evolution with respect to the algorithm iteration as given by Shi and Eberhart in [34]:where w max⁡ = 0.9 and w min⁡ = 0.4 represent the maximum and minimum inertia factor values, respectively, and k max⁡ is the maximum iteration number. Like the other metaheuristic method, the PSO algorithm is initially developed as an unconstrained optimizer. Finally, the basic PSO algorithm can be algorithmically understood from the following steps.

The parameters of the PSO to initialize search are to be defined, size n , maximum and minimum inertia weight values, c 1 and c 2 mostly equal to 2, and so forth.

Initialize the n particles by randomly generating the positions x 0 from the solution space and velocities v 0 are generally zero. Evaluate the initial population using the fitness function and determine the personnel best and local best.

Increment the iteration number k. For each particle apply the update equations (4) and (5), and evaluate the corresponding fitness values φ = φ(x ):

if φ ≤ pbest   then pbest = φ and p = x ;

if φ ≤ gbest then gbest = φ and p = x ,

where pbest and gbest represent the best previous fitness of the ith particle and the entire swarm, respectively.

If the termination criterion is satisfied, the algorithm terminates with the final solution. Otherwise, go to step 3.

However, these basic variants of PSO suffer from some common problems, which are quite apparent in several such stochastic optimization algorithms, for example, “curse of dimensionality” and tendency of premature convergence and hence getting stuck in local optima. Hence, some improved versions of PSOs have been very recently proposed to address some of these specific drawbacks. In this paper, we propose one such improved version of PSO algorithms, called the self-regulated learning PSO algorithm (henceforth called SRLPSO), which attempts to overcome the problem of both “curse of dimensionality” and tendency of premature convergence.

3.2. Mechanism for the Self-Regulated Learning PSO Algorithm

Let us revisit the velocity updating equation as given in (5). Generally the first term in the above equation is the previous controlled velocity. Second term is the cognitive component which is responsible for the particles own knowledge in the search space. The third term is called social component which is the real mechanism that decides the overall exploration and exploitation capacities of the proposed PSO algorithm.

Here p is the global best in the current iteration. This p is the position of the particle that will assume values that contribute the best fitness value in the current iteration. We replace p by P , where P will assume values from both best (global) position p and worst position p . Thus each component in P will take a place based on a self-regulated acceptance criterion. Thereby in the beginning of the runs, the majority of the components of P will be influenced by p , to establish exploration, and in long run with p as the algorithm proceeds to its final iterations to establish exploitation. The self-regulating PSO achieves faster convergence in comparison with conventional PSO due to the fact that the global best point will take values from both best position and worst position and each component updation will take a place based on a self-regulated acceptance criterion, thus establishing exploration in the initial iterations and proceeding towards final iterations to establish exploitation. Thereby the equation is rewritten asThe choice of each component in P is the main contribution in this research work and it is explained as follows: where p is the position of the particle that will assume values that contribute the worst fitness value in the current iteration. iter is the iteration count in the current iteration. This mechanism is proved to be a best diversifier as it is proved as a global mechanism for thoroughly exploring the solution space.

3.3. The Flowchart of the Proposed Self-Regulated Learning PSO (SRLPSO) Algorithm

The flowchart of the self-regulated learning scheme is shown in Figure 3.

Figure 3

Flowchart of the PSO with self-regulated learning scheme.

4. Proposed SRLPSO Based ELM for Medical Dataset Classification

This proposed methodology combines the concept of SRLPSO for optimizing the weights in ELM neural network. This self-regulated learning PSO with ELM enables the selection of input weights to increase the generalization performance and the conditioning of the single layer feedforward neural network. The steps of the proposed approach are as follows.

Step 1 .

Initialize positions with a set of input weights and hidden biases: [W 11, W 12,…, W 1,…, W 21, W 22,…, W 2,…, W , W ,…, W , b 1, b 2,…, b ]. These will be randomly initialized within the range of [−1,1] on D dimensions in the search space.

Step 2 .

For each member in the group, the respective output final weights are computed at ELM as given in (3).

Step 3 .

Now invoke self-regulated learning PSO as in Section 3.3.

Step 4 .

Then the fitness, which is the mean square error (MSE) of each member, is evaluated as where N is the number of training samples and the terms y and d are the error of the actual output and target output of the kth output neuron of ith sample. Thus, fitness function f(x) is defined by the MSE. In order to avoid overfitting of the single layer feedforward neural network, the fitness of each member is adopted as the mean squared error (MSE) on the validation set only instead of the whole training set as in [35].

Step 5 .

Find the SRL acceptance based on the fitness.

Step 6 .

Update the velocity and position equations of each particle as given in (4) and (5).

Step 7 .

Stopping criteria: the algorithm repeats Steps 2 to 6 until certain criteria are met, along with hard threshold value as maximum number of iterations. Once stopped, the algorithm reports values with optimal weights with minimal MSE as its solution.

Thus SRLPSO with ELM finds the best optimal weights W and bias b so that the fitness reaches the minimum to achieve better generalization performance, with minimum number of hidden neurons, considering both the advantages of both ELM and SRLPSO. In the process of selecting the input weights, the self-regulated learning PSO considers not only the MSE on validation set but also the norm of the output weights. The proposed SRLPSO based ELM will combine the feature of SRLPSO into ELM to compute the optimal weights and bias to make the MSE minimal.

5. Description of Medical Dataset from UCI Repository

The performance of the proposed SRLPSO-ELM method is experimented on five real benchmark classification problems (UCI Machine Learning Repository). The specification of these problems is listed in Table 1. The training, testing, and validation datasets are randomly regenerated at each trial of simulations according to Table 1 for the proposed SRLPSO-ELM algorithms. The five benchmark datasets on which evaluation results are carried out are Wisconsin Breast Cancer, Pima Indians Diabetes, Heart-Statlog, Hepatitis, and Cleveland Heart Disease, which are available from the UCI Machine Learning Repository [36]. Table 1 summarizes the number of features, instances, and classes for each dataset used in this study. All this data information is reproduced for the benefit of easy reference from [16].

Table 1

Description of datasets.

Dataset	Number of instances	Number of features	Number of classes
Wisconsin Breast Cancer	699	9	2
Pima Indians Diabetes	768	8	2
Heart-Statlog	270	13	2
Hepatitis	155	19	2
Cleveland Heart Disease	296	13	5

Wisconsin Breast Cancer [. The dataset was collected by Dr. William H. Wolberg (1989–1991) at the University of Wisconsin-Madison Hospitals. It contains 699 instances characterized by nine features: (1) clump thickness, (2) uniformity of cell size, (3) uniformity of cell shape, (4) marginal adhesion, (5) single epithelial cell size, (6) bare nuclei, (7) bland chromatin, (8) normal nucleoli, and (9) mitoses, which are used to predict benign or malignant growths. In this dataset, 241 (34.5%) instances are malignant and 458 (65.5%) instances are benign.

Pima Indians Diabetes [. The dataset is available at the National Institute of Diabetes and Digestive and Kidney Diseases. It contains 768 instances described by eight features used to predict the presence or absence of diabetes. The features are as follows: (1) number of pregnancies, (2) plasma glucose concentration, (3) diastolic blood pressure, (4) tricep skin fold thickness, (5) serum insulin, (6) body mass index, (7) diabetes pedigree function, and (8) age in years.

Heart-Statlog [. The dataset is based on data from the Cleveland Clinic Foundation and it contains 270 instances belonging to two classes: the presence or absence of heart disease. It is described by 13 features (age, sex, chest, resting blood pressure, serum cholesterol, fasting blood sugar, resting electrocardiographic, maximum heart rate, exercise induced angina, old peak, slope, number of major vessels, and thal).

Hepatitis [. The dataset is obtained from the Carnegie-Mellon University and it contains 155 instances belonging to two classes: live or die. There are 19 features (age, sex, steroid, antivirals, fatigue, malaise, anorexia, big liver, liver film, palpable spleen, spiders, ascites, varices, bilirubin, alk phosphate, SGOT, albumin, protime, and histology).

Cleveland Heart Disease [. The dataset was collected from the Cleveland Clinic Foundation and contains about 296 instances, each having 13 features (originally 76 raw features), which are used to infer the presence (values 1, 2, 3, and 4) or absence (value 0) of heart disease. The features are (1) age, (2) sex, (3) chest pain type, (4) resting blood pressure, (5) cholesterol, (6) fasting blood sugar, (7) resting electrocardiographic results, (8) maximum heart rate, (9) exercise induced angina, (10) depression induced by exercise relative to segment, (11) slope of peak exercise, (12) number of major vessels, and (13) thal.

6. Experimental Results and Discussion

The performance of the proposed SRLPSO-ELM method is experimented on five real benchmark classification problems (UCI Machine Learning Repository). Out of the data samples, 70% is employed for training process and 30% for testing process. The specification of these problems is listed in Table 2.

Table 2

Parameters for the proposed algorithms.

Parameters	ELM	Parameters	SRLPSO
Initial weights and bias between networks	0	Particle size	50
Learning rate	1	Acceleration constants c 1 and c 2	2.0
Number of neurons in input and output layers	Based on datasets considered	w min⁡ and w max⁡	0.4 and 1.0
Maximum iteration	500	Maximum iteration	500

On applying the proposed SRLPSO-ELM algorithm for the considered medical datasets from the repositories, the dataset features are selected and the parameters, classification accuracy, sensitivity, and specificity, are noted and are tabulated. Hence, in this case, the performance of the proposed method is evaluated in terms of sensitivity, specificity, and accuracy. In general, sensitivity and specificity are the statistical measures employed basically to carry out the performance of classification functions. A perfect classifier or a predictor will be described as 100% sensitive and 100% specific. Since the considered application is of medical dataset involving complex data, the classification should be carried out in an accurate manner. Henceforth, sensitivity and specificity are chosen to be the parametric indices for carrying out the medical dataset classification.

Sensitivity (true positive fraction) is the probability that a diagnostic test is positive, given that the person has the disease:Specificity (true negative fraction) is the probability that a diagnostic test is negative, given that the person does not have the disease:Accuracy is the probability that a diagnostic test is correctly performed: where TP (true positives) is correctly classified positive cases. TN (true negative) is correctly classified negative cases. FP (false positives) is incorrectly classified negative cases. FN (false negative) is incorrectly classified positive cases.

6.1. Breast Cancer Dataset

In Table 3, the results are reported for different feature selection methods for the breast cancer dataset. On classifying the dataset employing original features, it is noted that the classification accuracy of 95.85%, sensitivity of 0.92, and a specificity of 0.98 are obtained. When the PSO-ELM approach is applied an accuracy of 99.62%, sensitivity of 0.9961, and a specificity of 0.9893 are obtained, respectively. On applying the proposed SRLPSO and ELM approach, the accuracy is increased significantly to 99.78%. The best sensitivity and specificity of 1.00 are achieved with the SRLPSO and ELM approach. The highest accuracy is reported for this dataset when the proposed SRLPSO-ELM approach is employed. Figure 4 shows the classification rate for breast cancer dataset employing PSO and proposed SRLPSO with ELM classifier with respect to accuracy, sensitivity, and specificity.

Table 3

Classification results with Breast Cancer dataset.

Methodology adopted	Accuracy (%)	Sensitivity (%)	Specificity (%)	Selected features
Optimized LVQ (10x CV) [10]	96.70	91.29	92.34	2, 3, 6
Big LVQ (10x CV) [10]	96.80	95.23	96.10	2, 3, 6
AIRS (10x CV) [10]	97.20	96.92	95.00	2, 3, 6, 7
Supervised fuzzy clustering (10x CV) [11]	95.57	98.23	97.36	2, 3, 6, 7, 8
Fuzzy-AIS-knn (10x CV) [12]	99.14	99.56	100	2, 3, 6, 7, 8
F-score + support vector machine [13]	99.51	99.24	98.61	2, 3, 6, 7
Association rule + neural network [14]	97.4	93.12	91.26	2, 3, 6, 7, 8
Artificial metaplasticity neural network [15]	99.26	100	97.89	2, 3, 6, 7, 8
Mean selection method [16]	95.99	93	97	2, 3, 6, 7
Half selection method [16]	96.71	94	98	2, 3, 6, 7, 8
Neural network for threshold selection [16]	97.28	94	99	1, 2, 3, 5, 6, 7, 8
PSO + ELM	99.62	99.61	98.93	2, 3, 6, 7, 8
Proposed SRLPSO + ELM	99.78	100	100	2, 3, 6, 7

Figure 4

Classification rate for Breast Cancer dataset.

6.2. Pima Indians Diabetes Dataset

The performance of the different feature selection methods for the Pima Indians Diabetes dataset is shown in Table 4. It is noted that, on applying the basic PSO approach with that of ELM, in comparison with the other methods accuracy and the other parameters are increased to a value of 91.27% for accuracy and 0.8526 and 0.9410 for sensitivity and specificity. Employing the proposed SRLPSO with ELM mechanism has resulted in accuracy significantly increasing to 93.09%. Also in the case of proposed SRLPSO-ELM methodology, the sensitivity and specificity are noted to be 0.9147 and 0.9629, respectively, with that of only three features, wherein eight features were required in case of the original dataset. Figure 5 shows the classification rate for Diabetes dataset employing PSO and proposed SRLPSO with ELM classifier with respect to accuracy, sensitivity, and specificity.

Table 4

Classification results with Pima Indians Diabetes dataset.

Methodology adopted	Accuracy (%)	Sensitivity (%)	Specificity (%)	Selected features
PCA-ANFIS (10x FC) [17]	89.47	70	71.1	1, 2, 6, 7, 8
LS-ELM (10x FC) [18]	78.21	73.91	80	1, 2, 6, 8
GDA-LS-ELM (10x FC) [18]	79.16	79.1	83.33	1, 2, 6, 7, 8
MLNN with LM (10x FC) [19]	79.62	70	70.31	1, 2, 6, 8
PNN (10x FC) [19]	78.05	71	70.5	2, 6, 8
LDA-MWELM [20]	89.74	83.33	93.75	1, 2, 6, 7, 8
Mean selection method [16]	76.04	71	78	2, 6, 8
Half selection method [16]	75.91	69	79	1, 2, 6, 8
Neural network for threshold selection [16]	76.04	71	78	2, 6, 8
PSO + ELM	91.27	85.26	94.10	1, 2, 6, 8
Proposed SRLPSO + ELM	93.09	91.47	96.29	2, 6, 8

Figure 5

Classification rate for Diabetes dataset.

6.3. Heart-Statlog Dataset

Table 5 depicts the results obtained by the different feature selection methods with the Heart-Statlog dataset. The classification accuracy using the proposed SRLPSO-ELM has increased to 89.96% with three features. The required features were reduced drastically achieving better classification accuracy. Sensitivity is noted to be 0.8779 and specificity is 0.8842, comparatively better than all the other earlier existing techniques as well as that of PSO-ELM. The proposed SRLPSO-ELM achieves better classification accuracy rate than the earlier methods from the literature considered for comparison. Figure 6 shows the classification rate for Heart-Statlog dataset employing PSO and proposed SRLPSO with ELM classifier with respect to accuracy, sensitivity, and specificity.

Table 5

Classification results with Heart-Statlog dataset.

Methodology adopted	Accuracy (%)	Sensitivity (%)	Specificity (%)	Selected features
Evolutionary sigmoidal unit neural network (ESUNN) [21]	83.22	84.32	81.65	3, 8, 9, 11, 12
Evolutionary product unit neural network (EPUNN) [21]	81.89	83.67	84.91	8, 9, 11, 12
Multilogistic regression + EPUNN [22]	83.12	78.15	80.59	8, 9, 11, 12, 13
Mean selection method [16]	84.44	85	84	3, 8, 9, 11, 12, 13
Half selection method [16]	84.81	85	84	3, 8, 9, 10, 11, 12, 13
Neural network for threshold selection [16]	85.19	85	86	3, 11, 12, 13
PSO + ELM	85.88	86.00	86.03	3, 11, 12, 13
Proposed SRLPSO + ELM	89.96	87.79	88.42	11, 12, 13

Figure 6

Classification rate for Heart-Statlog dataset.

6.4. Hepatitis Dataset

The results for the Hepatitis dataset are shown in Table 6. From the results, it is inferred that the proposed SRLPSO-ELM approach yields a better accuracy of 98.71%, sensitivity 0.9427, and specificity 0.9604. Conventionally, with that of the original features an accuracy of 84.52%, sensitivity of 0.90, and specificity of 0.63 were noted. The proposed SRLPSO-ELM approach achieved the accuracy rate of 98.71% with six features. Figure 7 shows the classification rate for Hepatitis dataset employing PSO and proposed SRLPSO with ELM classifier with respect to accuracy, sensitivity, and specificity.

Table 6

Classification results with Hepatitis dataset.

Methodology adopted	Accuracy (%)	Sensitivity (%)	Specificity (%)	Selected features
Conventional artificial neural network [23]	97.00	92.31	94.5	All
Mean selection method [16]	82.58	87	60	5, 6, 11, 12, 13, 14, 17, 19
Half selection method [16]	85.16	90	66	2, 5, 6, 10, 11, 12, 13, 14, 17, 19
Neural network for threshold selection [16]	85.16	90	66	2, 5, 6, 10, 11, 12, 13, 14, 17, 19
PSO + ELM	97.43	93.65	95.71	6, 11, 12, 13, 14, 17, 19
Proposed SRLPSO + ELM	98.71	94.27	96.04	11, 12, 13, 14, 17, 19

Figure 7

Classification rate for Hepatitis dataset.

6.5. Cleveland Heart Disease Dataset

The results for the Cleveland Heart Disease dataset can be seen in Table 7. As a five-class classification problem is dealt instead of a binary-class classification problem, the Region of Convergence is omitted. It is to be noted that a classification accuracy of 83.82%, sensitivity of 0.83, and a specificity of 0.85 were noted when the original features of the dataset were considered. On applying the proposed SRLPSO-ELM approach, the best accuracy of 91.33% is achieved. This accuracy is achieved with only three features, compared with that of the 13 features of the original dataset. Figure 8 shows the classification rate for Cleveland Heart Disease dataset employing PSO and proposed SRLPSO with ELM classifier with respect to accuracy, sensitivity, and specificity.

Table 7

Classification results with Cleveland Heart Disease dataset.

Methodology adopted	Accuracy (%)	Sensitivity (%)	Specificity (%)	Selected features
C4.5 [31]	81.11	77.23	76.58	All
Naive Bayes [31]	81.48	80.97	81.22	3, 9, 11, 12
BNND [31]	81.11	82.13	80.42	3, 11, 12
BNNF [31]	80.96	76.93	75.81	3, 8, 9, 11, 12
AIRS [24]	84.50	75.34	72.96	All
Hybrid neural network [26]	87.40	93.00	78.50	3, 8, 9, 10, 11, 12
Neural networks ensemble [25]	89.01	80.95	95.91	All
Mean selection method [16]	81.75	82	82	3, 8, 9, 11, 12, 13
Half selection method [16]	83.44	84	83	3, 8, 9, 10, 11, 12, 13
Neural network for threshold selection [16]	84.46	82	82	3, 12, 13
PSO + ELM	89.47	94.49	96.02	9, 10, 11, 12, 13
Proposed SRLPSO + ELM	91.33	95.46	97.29	11, 12, 13

Figure 8

Classification rate for Cleveland Heart Disease dataset.

7. Conclusion

In this research a new hybrid algorithm that integrates the proposed self-regulated particle swarm optimization (SRPSO) algorithm with the extreme learning machine (ELM) for classification problems is presented. To optimize the input weights and hidden biases and minimum norm least-square scheme, an improved PSO is used to analytically determine the output weights. The PSO is enhanced by incorporating a mechanism which diversifies the search behavior of the particles so that the algorithm finds much better solutions. The performance of the proposed ELM framework using SRLPSO was better than the performance of the other methods reported in the literature for five benchmark datasets from the UCI Machine Learning Repository which are used for evaluation. The results also show that in the proposed framework the number of neurons in the hidden layer does not need to be selected by trial-and-error and the relevant input variables can be automatically selected, thus reducing the network size and improving the generalization capability.