Convolutional neural networks for the classification of chest X-rays in the IoT era.

Chest X-ray medical imaging technology allows the diagnosis of many lung diseases. It is known that this technology is frequently used in hospitals, and it is the most accurate way of detecting most thorax diseases. Radiologists examine these images to identify lung diseases; however, this process can require some time. In contrast, an automated artificial intelligence system could help radiologists detect lung diseases more accurately and faster. Therefore, we propose two artificial intelligence approaches for processing and identifying chest X-ray images to detect chest diseases from such images. We introduce two novel deep learning methods for fast and automated classification of chest X-ray images. First, we propose the use of support vector machines based on the AlexNet model. Second, we develop support vector machines based on the VGGNet16 method. Combined deep networks with a robust classifier have shown that the proposed methods outperform AlexNet and VGG16 deep learning approaches for the chest X-ray image classification tasks. The proposed AlexNet and VGGNet based SVM provide average area under the curve values of 98% and 97%, respectively, for twelve chest X-ray diseases.

Introduction

The Internet of things (IoT) allows the connection of X-ray machines in hospitals for the purpose of collecting big data. All hospitals contain X-ray machines, and these machines can be connected using IoT methods. Accessing all X-ray images means that massive amounts of data are produced. Then, this enormous amount of X-ray images can be modelled using artificial intelligence algorithms. Consequently, each hospital can connect to this central unit for the automated detection of diseases from X-ray images. Therefore, the IoT is a crucial tool in terms of the ability to instantly access data, and artificial intelligence is essential for modelling which enables diseases to be accurately detected on images. This work shows that the precise modelling of disease on X-ray images is possible by using deep learning models. Then this work will be extended for the multi-hospital setting.

Early treatment is crucial for people, as delays in treatment application can sometimes be fatal. Chest X-ray imaging enables the detection of pneumonia and other chest diseases. Figure 1 shows examples of healthy and pneumonia X-ray images.

Fig. 1

Normal (a) and Pneumonia (b) cases on Chest X-ray images. Images are taken from the Chest X-ray14 dataset [36]

Deep learning techniques [6, 9, 13, 31–33] have been utilized for object detection in images. ImageNet challenge [3] has shown that these deep learning techniques, which are mainly convolutional neural networks, can sometimes even provide better classification performance than humans. Among these techniques/models, ResNet-152 provides the best object recognition performances compared to other learning methods. Other deep convolutional neural networks include AlexNet [13], VGG [31], GoogleNet [32, 33], ResNet [6] and DensNet [9].

These models have also been applied to medical datasets [10, 11, 18, 19, 23–25]. These models allow the modelling of medical data for the detection and recognition of diseases. Serte and Demirel [23] proposed the Gabor wavelet-based deep learning method for the recognition of malignant melanoma and seborrheic keratosis skin lesions. This method decomposed the skin image into seven directional Gabor bands and then modelled each of the representations using AlexNet and ResNet-18 deep learning models. Furthermore, Serte and Demirel [23] utilized a wavelet-based technique for the detection of malignant melanoma and seborrheic keratosis skin lesions. Their approach is based on obtaining skin image into wavelet band and then modelling each image band using deep networks. Authors [20, 27] have also utilized CNNs for detecting COVID-19 on X-Ray images, while others [21, 28] have also employed CNNs to identify COVID-19 on CT-scans. Furthermore, authors [16, 17, 22, 26, 29] have shown that CNN models provide accurate results for eye disease. Therefore, CNN models can be used on different medical images for the diagnosis of disease types. A recent medical review paper [30] summarized the application of these models.

Recently, Jakub et al. [4] a convolutional neural network for classification of healthy, bacterial pneumonia, and viral pneumonia using X-ray images.

The proposed support vector machine-based AlexNet and VGG-16 models are different than the previously proposed AlexNet and VGG-16 models. AlexNet and VGG-16 models employ convolution, and the image representations are classified using a Softmax classifier. However, the proposed methods convolve images and then organize the convolution output using a support vector machine classifier (SVM). SVM is known as a more robust classifier than the Softmax classifier. As a result, using SMV in conjunction with AlexNet and VGG-16 improves chest X-ray image classification performance.

The main contributions of this work are as follows. First, we propose a novel support vector machine-based AlexNet deep learning model for chest X-ray classification. This method builds on convolving images using AlexNet architecture for feature modelling and then classifying these features using support vector regressions. Second, we propose a novel support vector machine based VGG-16 deep learning model for chest X-ray image classification. This method builds on convolving images using the VGG-16 model to obtain image representations and then learns these representations using support vector regressions.

The paper organizes the work in the following sections. First, related work provides a summary of the previous studies on skin lesion classification. Second, the proposed novel techniques are introduced. In addition, an associated dataset is explained. Finally, the two proposed approaches are evaluated using a dataset.

Related work

Wang et al. [36] used AlexNet, GoogleNet, VGG-16, and ResNet= 50 deep convolutional neural networks. The authors generated these models by retained ImageNet models on chest X-ray models. This work provided the classification of eight thorax diseases. The findings indicated that the ResNet-50 deep learning model outperformed all other deep learning models.

Yao et al. [38] proposed a multi-resolution based multi deep learning model. The authors used the ResNet model to reduce chest X-ray images and then they used a DensNet model to classify chest X-ray images in different resolutions. Wang and Xia [37] used classification and an attention-based deep learning model. The authors named this model the ChestNet model. The classification part of the ChestNet model contained a ResNet-152 convolutional neural network model. The output of the classification map of this model was further modelled using convolutions for more accurate classification. This work compared the proposed model with the classic ResNet model on a ChestX-ray14 dataset. The performance evaluation of the Chestnet model showed that Chestnet outperformed the ResNet model.

Gundel et al. [5] used a DensNet-121 convolutional neural network for the classification of 12 chest X-ray images. The authors created this network by adapting the ImageNet pre-trained DensNet-121 model to chest images. The proposed method provided higher overall accuracy than the AlexNet, GoogleNet, VGG-16, and ResNet= 50 deep networks.

Rajpurkar et al. [15] applied a DensNet-121 convolutional neural network for the detection of pneumonia. The authors also used this model for the detection of twelve thorax diseases. The authors trained the DensNet-121 model using a Chest X-ray14 dataset [36]. Li et al. [14] utilized an attention-based ResNet convolutional neural network. Their approach applied a U-Net deep learning segmentation model and then local regions of the X-ray images were classified using the ResNet model. Kermany [12] et al. proposed transfer learning based convolutional neural networks for Pneumonia detection. Varshni [35] et al. combined a DenseNet-169 model and support vector machines (SVM). Chest X-ray appearances are represented and related feature vectors are extracted using DenseNet-169 deep networks. The image representations are obtained by passing the architecture consisting of 169 layers. Then, the image appearances are classified using SVM for healthy and pneumonia images.

Ayan and Unver [1] utilized Xception and VGG-16 deep networks for the classification of pneumonia and healthy X-ray images. Their approach was based on retraining previously existing models for X-ray images. In other words, they used transfer learning-based models for the detection of pneumonia among pneumonia and healthy X-ray images. The VGG-16 model outperformed the Xception model for the classification task.

The proposed methods

Figures 2 and 3 present the proposed AlexNet and VGGNet16 deep learning models in conjunction with support vector machines. Classic deep learning models utilize Soft-max function for the classification of the images. In this work, we modify the deep model architectures and replace the Soft-max function with a multi-class support vector machines classifier, [2, 34]. SVM builds on margin-based loss minimization. The SVM model also employs regularization during modelling of the data samples. Therefore, the SVM model provides more accurate sample classification than the Soft-max based model. As a result, the proposed deep learning architecture employs feature extraction and then these models utilize powerful multi-class support vector machines to classify lung disease. The two proposed models allow twelve lung disease to be classified.

Fig. 2

The proposed SVM-AlexNet Method

Fig. 3

The proposed SVM-VGGNet16 Method

Image processing

The chest X-ray images are RGB images, and these image sizes are 256x256. The model extracts 224x224x3 random image patches. Then, these image patches are used as inputs to the proposed multi-class support vector machine based AlexNet and VGGNet16 models.

The proposed SVM-AlexNet method

Architecture

Figure 2 shows the proposed SVM-AlexNet architecture. This architecture consists of five convolution layers (Conv1, Conv2, Conv3, Conv4, and Conv5), two fully connected layers (FC1 and FC2) and a multi-class support vector machine model (SVM). The proposed method utilizes five convolutional layers to extract features of chest X-ray images. Then, these features are classified into twelve throat diseases using a multi-class SVM.

Modelling

We use a set of chest X-ray images as input to the AlexNet network. Input images are convolved with 96 filters with a size of 11x11 in the first convolution layer. Subsequently, the output of the first convolutional layer is used as the input for the second convolutional layer. Then, the pooling is performed on the production of the second convolutional layer. This input is filtered with 256 kernels of size 5x5 in the second convolutional layer. This convolution process is done for the third, fourth and fifth convolutional layers. Then, fully connected layers are constructed.

Multi-class support vector machines

We used multi-class support vector machines to map feature vectors of the convolutional layers to lung disease [7, 8]. We used a one-vs-rest approach to obtain multi-class support vector machines. There are mainly one-versus-one and one-versus-rest approaches for extending SVM to multi-class. We used the one-vs-rest approach since it is simple to use and computationally less expensive. We trained the proposed multi-class support vector machines as follows. All chest X-ray images are passed through the trained AlexNet model, and corresponding feature vectors of the chest X-ray images are obtained. These feature vectors are retrieved from the FC2 layer of the trained Alexnet model. The dimension of these feature vectors is 1x4096. Then, these feature vectors are fed into the multi-class support vector machines for classifying twelve throat disease. Section 3.4 describes support vector machines for binary classification.

The proposed SVM-VGGNet16 method

Figure 3 shows the proposed SVM-VGGNet16 architecture. This architecture consists of thirteen convolution layers (Conv1-Conv13), two fully connected layers (FC1-FC2) and a multi-class support vector machine model (SVM). The proposed method employs thirteen convolutional layers to model and extract features of chest X-ray images. Then, the extracted features are classified into 12 throat diseases using multi-class SVM in Section 3.2.3. The thirteen convolutional layers are responsible for the representation of chest X-ray images together with two fully connected layers (FC1 and FC2).

First, we train the VGGNet6 network using chest X-ray images to create a model. During training, input images go through twelve convolutional layers and then three fully connected layers. Twelve convolutional layers are generated using filter sizes of 3x3. However, the number of filters is different for each of the convolution layers. The number of filters used in the first and second convolutional layer is 64. Furthermore, the third and fourth layers are obtained using 128 filters. Moreover, the fifth and size convolutional layers utilize 256. Then 512 filters are used for the last size layers.

We use training images of chest X-ray images as input for the thirteen convolution layers and two fully connected layers. Then, we obtain 4096 dimensional vectors from the FC2 layer of the model. These vectors are used as inputs for the multi-class support vector machines to predict twelve lung diseases.

Support vector machine

The appearances of chest X-ray image deformations in Sections 3.2 and 3.3 are obtained using AlexNet and VGG-16 deep networks and then classified using support vector machines. Chest X-ray images appearances are denoted by x,..x and corresponding chest X-ray types are denoted by y,..y. After, given a chest X-ray appearance x, the corresponding chest X-ray image’s name is predicted using support vector machines, [2, 34], where f(x) denotes the estimated chest X-ray image. Weights and bias are also denoted by w,b, respectively. The weight and bias values are estimated by optimizing the following optimization equation. ϕ(x) transfers data points to other dimensions. Slack variables are denoted by ξ, and these variables guide observations towards the margin. C defines regularization.

The assumed dataset and performance metrics

ChestX-ray14 dataset, [36], is one of the largest datasets related to chest X-ray images. This database contains fourteen thorax disease. Table 1 shows fourteen thorax disease and the corresponding number of images for each of the diseases. The proposed SVM based AlexNet and VGG-16 deep models are evaluated for fourteen thorax diseases using this database.

Table 1

Thorax diseases and number of images

Thorax disease	No. of Images
Atelectasis	11167
Cardiomegaly	12071
Effusion	7646
Infiltration	16316
Mass	10042
Nodule	6480
Pneumonia	9836
Pneumothorax	4693
Consolidation	4544
Edema	8875
Emphysema	10070
Fibrosis	10380

We use accuracy, sensitivity, and specificity as performance evaluation metrics. These metrics can be described as follows. We denote true positive, positive, true negative, false positive, and false negative as TP, TN, FP, and FN, respectively.

Performance evaluation

We report the performance results of the proposed methods for twelve chest diseases in Table 3. Table 3 shows the area under the curve values (AUC).

Table 3

The performance comparison of the proposed models (AUC)

Thorax disease	AlexNet+SVM	VGG16+SVM	Wang et al. [36]	Yao et al. [38]	Gundel et al.	ResNet [37]	ChestNet [37]	CheXNet [15]
Atelectasis	0.97	0.97	0.70	0.73	0.76	0.69	0.74	0.80
Cardiomegaly	0.96	0.98	0.81	0.85	0.88	0.80	0.87	0.92
Effusion	0.98	0.98	0.75	0.80	0.82	0.77	0.81	0.86
Infiltration	0.98	0.97	0.66	0.67	0.70	0.64	0.67	0.73
Mass	0.98	0.97	0.69	0.71	0.82	0.71	0.78	0.86
Nodule	0.97	0.98	0.66	0.77	0.75	0.67	0.69	0.78
Pneumonia	0.98	0.98	0.65	0.68	0.73	0.63	0.69	0.76
Pneumothorax	0.98	0.99	0.79	0.80	0.84	0.77	0.80	0.88
Consolidation	0.98	0.96	0.70	0.71	0.74	0.69	0.72	0.93
Edema	0.97	0.97	0.80	0.80	0.83	0.80	0.83	0.80
Emphysema	0.98	0.98	0.83	0.84	0.89	0.79	0.79	0.80
Fibrosis	0.98	0.96	0.78	0.74	0.82	0.78	0.78	0.91
Avg	0.98	0.97	0.73	0.76	0.79	0.72	0.76	0.83

AlexNet + SVM column values is bold in Table 3

VGG16 + SVM column values is bold Table 3

We trained proposed methods using 70% of the ChestX-ray dataset, and we evaluated the network performances using 40% of the chest X-ray dataset.

Evaluation of The proposed models

Table 2 shows the accuracy, sensitivity and specificity values of proposed methods and classic deep learning methods. The proposed AlexNet+SVM and VGG16+SVM methods provide higher accuracy values that classic AlexNet and VGG16+SVM methods.

Table 2

The accuracy, sensitivity and specificity values of our proposed methods

Proposed Method	AC	SE	SP
AlexNet	0.94	0.99	0.94
VGG16	0.95	0.96	0.95
AlexNet+SVM	0.96	0.96	0.96
VGG16+SVM	0.98	0.99	0.98

AlexNet + SVM AC value (0.96) is bold in Table 2

VGG16 + SVM AC value (0.98) is bold Table 2

Figure 4 (a),(b),(c), and (d) also show confusion matrices of the AlexNet, VGG-16, and SVM based AlexNet and VGG-16 models. These matrices show both the number of true positive disease detections and accuracy values of the twelve chest disease. The overall accuracy values of the AlexNet, VGG-16, AlexNet+SVM, and VGG-16+SVM models are 94.1%, 95.3%, 96.3%, and 98.1%. The proposed AlexNet+SVM and VGG-16+SVM models provide higher accuracy than classic deep learning models. Moreover, the AlexNet-based SVM provides an average AUC value of 98% and the VGGNet-based SVM provides an average AUC value of 97% for twelve chest X-ray diseases.

Fig. 4

Chest X-ray image in ChestX-ray8 dataset

Comparison of the proposed method and other methods

Table 3 shows the performances for AlexNet, AlexNet+SVM, VGG-16, and VGG-16+SVM models. The AUC values of the models are similar.

AlexNet, GoogleNet, VGG-16, ResNet-50, DensNet, and attention-based deep learning-based methods were utilised for the classification of X-ray images. Previous performance results have shown that the ResNet-50 deep network provides higher accuracy than AlexNet, GoogleNet, and VGG-16 [36]. More recent studies show that DensNet architecture [15] further improves the performance of the chest X-ray image classification. The proposed support vector machine-based deep network has shown that combining AlexNet and VGG-16 networks with support vector machines improve the classic AlexNet and VGG-16 networks. The performance results indicate that classifying chest X-ray images using SVM instead of the Softmax classifier results in a more powerful learning algorithm.

Discussion

Accuracy

The proposed methods outperform other techniques for the classification of the twelve throat disease. Previous works have utilized deep learning methods with the Softmax function for classification. In contrast, the proposed approaches employ deep learning models in conjunction with support vector machines (SVMs) for classification. Softmax is based on cross-entropy minimization while the data samples are classified. On the other hand, an SVM builds on margin-based loss minimization. The SVM model also employs regularization during modelling of the data samples. Therefore, the proposed SVM provides a more accurate sample classification than the use of only cross-entropy loss minimization. Thus, the proposed approaches outperform current works for the detection of twelve throat diseases.

Computational complexities

Rajpurkar et al. [15] employed DensNet-121 and Wang et al., [37] use ResNet-152 deep networks. These networks convolve images to create 121 and 151 convolutional layers, respectively. In contrast, the proposed AlexNet and VGGNet16 convolve images to create 8 and 16 layers. Since the proposed methods include fewer layers than the DensNet-121 and ResNet-152 models, the models are more computationally efficient. Furthermore, novel methods employ SVM models; therefore, the models are more accurate than the DenseNet-121 and ResNet-152 modes.

AlexNet computational complexity

Table 4 provides the number of convolution operations in each of the five convolutional layers of the AlexNet network. We also provide the total number of convolution operation for five convolutional layers.

Table 4

AlexNet computational complexity

Conv. layer	Number of filters	Filter size	Stride	Padding	Number of convolutions
1.Layer	64	11x11	4x4	2x2	3472
2.Layer	192	5x5	1x1	2x2	42816
3.Layer	384	3x3	1x1	1x1	85632
4.Layer	256	3x3	1x1	1x1	57088
5.Layer	256	3x3	1x1	1x1	57088
Total	–	–	–	–	246096

VGG16 computational complexity

Table 5 provides the number of convolution operations of thirteen convolutional layers of the VGG16 network. We also provide the total number of convolution operation for five convolutional layers.

Table 5

VGG16 computational complexity

Conv. Layer	Number of filters	Filter size	Stride	Padding	Number of convolutions
1.Layer	64	3x3	1x1	1x1	14272
2.Layer	64	3x3	1x1	1x1	14272
3.Layer	128	3x3	1x1	1x1	28544
4.Layer	128	3x3	1x1	1x1	28544
5.Layer	256	3x3	1x1	1x1	57088
6.Layer	256	3x3	1x1	1x1	57088
7.Layer	256	3x3	1x1	1x1	57088
8.Layer	512	3x3	1x1	1x1	114176
9.Layer	512	3x3	1x1	1x1	114176
10.Layer	512	3x3	1x1	1x1	114176
11.Layer	512	3x3	1x1	1x1	114176
12.Layer	512	3x3	1x1	1x1	114176
13.Layer	512	3x3	1x1	1x1	114176
Total	–	–	–	–	941952

Advantages and disadvantages

One of the advantages of the proposed methods is that a support vector machine (SVM) can easily be integrated into a deep learning architecture. The trained SVM model retrieves feature vectors from the last fully connected deep learning model, and it provides class scores. Another advantage is that using SVM instead of the Softmax classifier allows better classification accuracy for disease detection. However, the proposed methods also have certain disadvantages. Deep learning models, in conjunction with SVM models, require two training tasks. First, deep learning models are trained, and then the models are created. The generated models are used to extract features from the images. Second, extracted features are used as inputs to the SVM model for SVM model training. However, classic deep networks are only required in the training step.

Limitations

We only use one dataset to compare the proposed methods with other methods. Using one dataset might not show whether the models are generalizable or not. Models can be tested on more than one dataset to evaluate them in terms of generalizability. Another limitation is that running models and SVM models might be problematic for mobile phones or web servers. The reason is that the available hardware memory capacity of these devices is limited.

Conclusion

This study presents two new support vector machines based on AlexNet and VGG-16 deep learning models for the classification of chest X-ray images. First, AlexNet architecture is proposed to model skin image appearances, and then support vector machines are used to classify these appearances. This method is utilized to support vector machines instead of the Softmax method for more robust and accurate chest X-ray image classification. Then, we developed another support vector machine-based VGG-16 deep learning method. Similarly, this method convolves images for extracted features and classifies skin lesions using support vector machines. The proposed methodologies replace the Softmax layer with SVM. Since support vector machines are more powerful classifiers than Softmax, the proposed methods provide higher accuracy compared to classic methods. The performance results show that the proposed models outperform the typical deep learning AlexNet and VGG-16 networks.