A novel study for automatic two-class COVID-19 diagnosis (between COVID-19 and Healthy, Pneumonia) on X-ray images using texture analysis and 2-D/3-D convolutional neural networks.

The pandemic caused by the COVID-19 virus affects the world widely and heavily. When examining the CT, X-ray, and ultrasound images, radiologists must first determine whether there are signs of COVID-19 in the images. That is, COVID-19/Healthy detection is made. The second determination is the separation of pneumonia caused by the COVID-19 virus and pneumonia caused by a bacteria or virus other than COVID-19. This distinction is key in determining the treatment and isolation procedure to be applied to the patient. In this study, which aims to diagnose COVID-19 early using X-ray images, automatic two-class classification was carried out in four different titles: COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia. For this study, 3405 COVID-19, 2780 Bacterial Pneumonia, 1493 Viral Pneumonia, and 1989 Healthy images obtained by combining eight different data sets with open access were used. In the study, besides using the original X-ray images alone, classification results were obtained by accessing the images obtained using Local Binary Pattern (LBP) and Local Entropy (LE). The classification procedures were repeated for the images that were combined with the original images, LBP, and LE images in various combinations. 2-D CNN (Two-Dimensional Convolutional Neural Networks) and 3-D CNN (Three-Dimensional Convolutional Neural Networks) architectures were used as classifiers within the scope of the study. Mobilenetv2, Resnet101, and Googlenet architectures were used in the study as a 2-D CNN. A 24-layer 3-D CNN architecture has also been designed and used. Our study is the first to analyze the effect of diversification of input data type on classification results of 2-D/3-D CNN architectures. The results obtained within the scope of the study indicate that diversifying X-ray images with tissue analysis methods in the diagnosis of COVID-19 and including CNN input provides significant improvements in the results. Also, it is understood that the 3-D CNN architecture can be an important alternative to achieve a high classification result.

Introduction

The COVID-19 virus emerged in China in December 2019 and quickly spread to other countries. As a result of this rapid spread, the World Health Organization (WHO) announced in March that the disease had developed into a pandemic [1]. The pandemic still affects many countries deeply. Even in countries with developed health systems, there are occasional infrastructural deficiencies [2]. The main reason for this situation is the increase in the need for intensive care and respiratory support due to pneumonia caused by the virus [3]. As of the last week of November 2020, the number of people infected with the virus has reached 60 million. The number of deaths related to the virus approached 1.5 million [4]. Vaccine development studies against the COVID-19 virus are still ongoing. However, it is understood that the pandemic will affect people and countries economically, socially, and politically for a long time.

The most common symptoms of the COVID-19 virus are fever, shortness of breath, and pneumonia [5]. The diagnosis of the COVID-19 virus is carried out with reverse transcription-polymerase chain reaction (RT-PCR). Generally, this test produces a result within a few hours [6]. One of the most important factors in controlling the COVID-19 outbreak is the rapid diagnosis and isolation of patients carrying the virus. The RT-PCR test takes hours to produce results, making it difficult to quickly isolate individuals suspected of carrying the virus. In addition, since the human factor in swabbing for this test is high, there are some problems producing the correct result. For this reason, the WHO requires not only the direct reporting of patients with positive RT-PCR tests but also the cases that are detected to carry this virus by radiological or clinical symptoms despite a negative RT-PCR result [7].

In academic studies on COVID-19, it has been shown that the virus causes interstitial involvement, lung opacities, bilateral ground-glass, and patchy opacity in the lungs [8]. This situation means that radiological imaging methods, such as CT, X-ray, and ultrasound, are important tools in the diagnosis of the COVID-19 virus. When they examine the CT, X-ray, and ultrasound images, the first job radiologist do is to determine whether there are signs of COVID-19 in the images. That is, COVID-19/Healthy detection is made. Then, they determine whether it is the separation of pneumonia caused by the COVID-19 virus or pneumonia caused by a bacteria or virus other than COVID-19. The result of this distinction is an important indicator when determining the treatment and isolation procedure to be applied to the patient. This distinction can be described as the COVID-19 Pneumonia/Bacterial, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia classification. In the academic studies carried out, the different aspects of pneumonia caused by the COVID-19 virus from other types of pneumonia have been presented in detail [9].

Automated inspection of CT, X-ray, and ultrasound images is an important requirement due to the heavy workload imposed on the healthcare system and staff by the outbreak [10]. In addition, it is an important auxiliary factor in controlling the pandemic as it will speed up the detection of COVID-19.

When the methods used in the studies given in Sect. 2 are examined in detail, they generally suggest different CNN architectures for classification. In this context, it is understood that they generally focus on revealing the effects of CNN architecture change on study results. In this study, the effect of diversification of CNN input images using texture feature methods instead of the CNN architecture used differently from previous studies was investigated. In this context, CNN architecture, which provides the best results in previous studies, was added to the study.

In this study aiming at the early diagnosis of COVID-19 disease using X-ray images, an automatic two-class classification in four different titles as COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia has been carried out. Within the scope of the study, 3405 COVID-19, 2780 Bacterial Pneumonia, 1493 Viral Pneumonia, and 1989 Healthy images obtained by combining eight different data sets with open access were used. In the study, besides the direct use of the original X-ray images alone, classification results were obtained using the images resulting from Local Binary Pattern (LBP) and Local Entropy (LE). The classification processes were repeated for the images obtained by combining these images in various ways. Classifiers used within the scope of the study were 2-D CNN (Two-Dimensional Convolutional Neural Networks) and 3-D CNN (Three-Dimensional Convolutional Neural Networks) architectures. As a 2-D CNN, Mobilenetv2, Resnet101, and Googlenet architectures were used in the study. As a 3-D CNN, a 24-layer 3-D CNN architecture has been designed. The results of the study indicate that as a result of the proposed methods, significant gains can be achieved in the diagnosis of COVID-19 using X-ray images.

It is possible to summarize the innovative aspects and importance of our work with the following main headings.

Within the scope of our study, experiments were carried out with several images that have never been used in any study before. In this context, experiments were conducted with images approximately ten times more than the average number of COVID-19 images used in previous studies in our research. In addition, unlike other literature studies, more than 90% of the images in question are taken from the real world.

CNN architectures, which provide the highest results in previous studies, have also been added to our study. Experiments were carried out to diversify the input data for the mentioned CNN architectures. In this way, much more comprehensive and higher results have been obtained.

In addition to 2-D CNN architectures, similar operations were performed for a 3-D CNN architecture. The effects of diversifying the input data of the 3-D CNN architecture on the study results were revealed. In addition, the achievements of 2-D CNN architectures and 3-D CNN architectures are extensively compared.

Related works

Some academic studies have been carried out to meet this need by disseminating and sharing radiological data on patients with COVID-19. In this context, two-class classification studies, carried out in 2020 and focused on the COVID-19/Healthy distinction, were examined.

In the COVID-19/Healthy classification study performed by Tuncer et al. [11], a total of 321 X-ray images, including 87 COVID-19 and 234 Healthy, were used. In the study in question, Residual Exemplar Local Binary Pattern, Iterative Relief, Decision Tree, Linear Discriminant, Support Vector Machine, k-Nearest Neighborhood, and Subspace Discriminant methods were used in classification procedures. In the study, train-test procedures were carried out using three different methods: tenfold cross-validation, 80% Train-20% Test, and 50% Train-50% Test. At the end of the study, sensitivity values varying between 0.8149 and 1.0000, specificity varying between 0.9380 and 1.0000, and accuracy values varying between 0.9049 and 0.9955 were obtained.

Panwar et al. [12] conducted a COVID-19/Healthy classification study with a total of 284 X-ray images, including 142 COVID-19 and 142 Healthy, using the Convolutional Neural Network (CNN) (nCOVnet). The sensitivity was 0.9762, specificity was 0.7857, and accuracy was 0.881 as a result of training and testing using the 70% Train-30% Test.

In the classification study performed by Ozturk et al. [13], 625 X-ray images (125 COVID-19 and 500 Healthy) were used. In the study in which CNN (DarkNet) was used as a classifier, training–testing procedures were carried out using a fivefold cross-validation approach. At the end of the study, sensitivity was 0.9513, specificity 0.953, accuracy 0.9808, and F-1 Score 0.9651.

Mohammed et al. [14] proposed a decision tree mechanism for COVID-19/Healthy classification. The decision tree in question is based on combining the results obtained using Naive Bayes, Neural Network, Support Vector Machine, Radial Basis Function, k-Nearest Neighbors, Stochastic Gradient Descent, Random Forests, Decision Tree, AdaBoost, and CN2 Rule Inducer Algorithm methods. As a result of the study performed using a total of 50 X-ray images, including 25 COVID-19 and 25 Healthy, sensitivity values varying between 0.706 and 0.974 and specificity values varying between 0.557 and 1.000 were obtained. In addition, accuracy values ranging from 0.620 to 0.987, F-1 Score values varying between 0.555 and 0.987, and Area Under Receiver-Operating Characteristic Curve (AUC) values varying between 0.800 and 0.988 were provided. In the study, the run time was also measured, and results were obtained at times ranging from 0.14 to 7.57 s per image.

In a study conducted by Toraman et al. [15], a total of 1281 X-ray images were used, including 231 COVID-19 and 1050 Healthy. The training and testing procedures of the study, in which the CNN (CapsNet) was used as a classifier, were carried out based on the tenfold cross-validation method. At the end of the study, sensitivity values varying between 0.28 and 0.9742, specificity values varying between 0.8095 and 0.98, accuracy values varying between 0.4914 and 0.9724, and F-1 Score values varying between 0.55 and 0.9724 were provided. In the study, the run time per image was measured at values ranging from 16 to 500 s.

In the classification study performed by Khan et al. [16] using 594 X-ray images (284 COVID-19 and 310 Healthy), the CNN [CoroNet (Xception]) classifier was used. As a result of the training and testing processes performed using the fourfold cross-validation method, sensitivity was calculated as 0.993, specificity 0.986, accuracy 0.990, and F-1 Score 0.985.

In the COVID-19/Healthy classification study carried out by Yaşar and Ceylan [17], two datasets containing 230 X-ray images (150 COVID-19 and 80 Healthy) and 476 X-ray images (150 COVID-19 and 326 Healthy) were used. In addition, the classification process was repeated by combining these two datasets (556 X-ray images [150 COVID-19 and 406 Healthy]). In the study, two different CNN architectures, 23-layer and 54-layer, were used with Local Binary Pattern. The Dual-Tree Complex Wavelet Transform was used to reduce the image dimensions. In addition, in the study, pipeline algorithms that enable combining the results obtained using or not using Local Binary Pattern are proposed. As a result of the training and testing processes performed using the 23-fold cross-validation and twofold cross-validation approach, sensitivity values varying between 0.9253 and 0.9947, specificity varying between 0.9725 and 1.0000, and accuracy values varying between 0.9745 and 0.9906 were obtained. In addition, F-1 Score values ranging from 0.9511 to 0.9881, and AUC values ranging from 0.9975 to 0.9997 were provided.

In the study conducted by Waheed et al. [18], CNN (VGG-16) and Synthetic Data Augmentation methods were used. A total of 1124 X-ray images (403 COVID-19 and 721 Healthy) were divided into two parts for training (932 X-ray images [331 COVID-19 and 601 Healthy]) and testing (192 X-ray images [72 COVID-19 and 120 Healthy]). As a result of the training-test processes, sensitivity values varying between 0.69 and 0.90, specificity between 0.95 and 0.97, and accuracy values varying between 0.85 and 0.95 were achieved. Duran-Lopez et al. [19] used CNN as the classifier in their COVID-19/Healthy classification study using a total of 6926 X-ray images, including 2589 COVID-19 and 4337 Healthy. As a result of the training and testing procedures performed using the fivefold cross-validation approach, sensitivity was 0.9253, specificity 0.9633, accuracy 0.9443, F-1 Score 0.9314, and AUC 0.988.

Vaid et al. [20] proposed using CNN (VGG-19) and Trainable Fully Connected Layers methods for classifying COVID-19/Healthy. In the scope of the study, 545 images (181 COVID-19 and 364 Healthy) were divided into three parts: 348 images (115 COVID-19 and 233 Healthy) for training, 88 images (32 COVID-19 and 56 Healthy) for validation, and 109 images (34 COVID-19 and 75 Healthy) for testing. At the end of the training and testing procedures, sensitivity was 0.9863, specificity 0.9166, accuracy 0.9633, and F-1 Score 0.9729.

In the study conducted by Benbrahim et al. [21], a total of 320 X-ray images were used, including 160 COVID-19 and 160 Healthy. The training and testing procedures of the study in which Transfer Learning with Convolutional Neural Networks (Inceptionv3 and ResNet50) were used as a classifier were performed according to the 70% Train-30% Test procedure. At the end of the study, sensitivity ranging between 0.9803 and 0.9811, accuracy varying between 0.9803 and 0.9901, and F-1 Score values varying between 0.9803 and 0.9901 were obtained.

In the study conducted by Loey et al. [22], Transfer Learning with CNN (Alexnet, Googlenet, and Resnet18) methods were used for classification. Within the scope of the study, 148 images (69 COVID-19 and 79 Healthy) were divided into two parts: 130 images for training (60 COVID-19 and 70 Healthy), and 18 images for testing (nine COVID-19 and nine Healthy). At the end of the study, sensitivity, specificity, and accuracy values were obtained as 1.000.

Minaee et al. [23] suggested using Transfer Learning with CNN (ResNet18, ResNet50, SqueezeNet, and DenseNet-121) methods for COVID-19/Healthy classification. While performing the training and testing processes of the study, 5184 images (184 COVID-19 and 5000 Healthy) were divided into two parts: 2084 images (84 COVID-19 and 2000 Healthy) for training and 3100 images (100 COVID-19 and 3000 Healthy) for testing. At the end of the study, sensitivity was obtained as 0.98. Specificity values ranging from 0.751 to 0.929 have also been calculated.

In the study conducted by Elaziz et al. [24], two different datasets containing 1891 images (216 COVID-19 and 1675 Healthy) and 1560 images (219 COVID-19 and 1341 Healthy) were used. In the study where classification processes were performed using Fractional Multichannel Exponent Moments, Manta-Ray Foraging Optimization, and k-NN classifier, training and testing procedures were carried out according to 80% Train-20% Test procedures. At the end of the study, sensitivity values ranging from 0.9875 to 0.9891 and accuracy values between 0.9609 and 0.9809 were achieved.

Martínez et al. [25] conducted a study using the CNN (Neural Architecture Search Network [NASNet]) classifier with 240 images (120 COVID-19 and 120 Healthy) to classify COVID-19/Healthy. As a result of the training and testing performed with the 70% Train-30% Test procedure, the sensitivity was 0.97, the accuracy 0.97, and the F-1 score was 0.97.

In the study conducted by Mahmud et al. [26] using 610 images (305 COVID-19 and 305 Healthy) and Transfer Learning with CNN (Stacked Multi-Resolution CovXNet) classifier, training and testing procedures were carried out with a fivefold cross-validation approach. At the end of the study, sensitivity was 0.978, and specificity was 0.947. The accuracy was calculated as 0.974, F-1 Score 0.971, and AUC 0.969.

Next, in this section, two-class classification studies were examined. These were conducted in 2020, focusing on the distinctions of COVID-19 Pneumonia/Viral Pneumonia, COVID-19 Pneumonia/Bacterial Pneumonia, and COVID-19 Pneumonia/Other Pneumonia.

In the study conducted by Mahmud et al. [26], the classification of COVID-19 Pneumonia/Viral Pneumonia and COVID-19 Pneumonia/Bacterial Pneumonia was also performed. Within the scope of the study, a total of 610 X-ray images (305 COVID-19 and 305 Viral Pneumonia) were used for the differentiation of COVID-19 Pneumonia/Viral Pneumonia. Similarly, for the differentiation of COVID-19 Pneumonia/Bacterial Pneumonia, a total of 610 X-ray images (305 COVID-19 and 305 Bacterial Pneumonia) were used. In the study, where Transfer Learning with CNN (Stacked Multi-Resolution CovXNet) was used as a classifier, training and testing procedures were carried out according to the fivefold cross-validation procedure. As a result of the study, the sensitivity for COVID-19 Pneumonia/Viral Pneumonia classification was calculated as 0.874, specificity 0.855, accuracy 0.873, F-1 Score 0.878, and AUC 0.921. For the COVID-19 Pneumonia/Bacterial Pneumonia classification, sensitivity was 0.944, specificity 0.933, accuracy 0.947, F-1 Score 0.939, and AUC 0.951.

Horry et al. [27] proposed Transfer Learning with CNN (VGG-16, VGG-19, Xception, Inception ResNet, InceptionV3, NASNetLarge, Resnet 50V2, and DenseNet-121) methods for COVID-19 Pneumonia/Other Pneumonia classification. The study used 329 images (139 COVID-19 and 190 Other Pneumonia). Training and testing procedures were performed according to the 80% Train-20% Test procedure. At the end of the study, sensitivity ranging from 0.86 to 0.89 and F-1 Score values varying between 0.86 and 0.89 were provided.

If a general analysis of the number of X-ray images used in the literature studies with detailed descriptions above is made, it is understood that 19 data sets used in 16 studies for COVID-19/Healthy classification contain an average of 304.7 COVID-19 and 903.9 Healthy images. Similarly, 305 COVID-19 and 305 Viral Pneumonia images were used for COVID-19 Pneumonia/Viral Pneumonia classification, 305 COVID-19 and 305 Bacterial Pneumonia images were used for COVID-19 Pneumonia/Bacterial Pneumonia classification, and 139 COVID-19 and 190 Other Pneumonia images were used for COVID-19 Pneumonia/Other Pneumonia classification, respectively. The average number of COVID-19 X-ray images used in the studies is around 300, which is acceptable for initial stage studies. However, it is still important to carry out studies using a much larger number of images to increase the reliability and stability of the results. In addition, the COVID-19 images used in these studies are generally metadata data collected from academic studies by Cohen et al. [28]. The use of real-world images in studies is a basic requirement for testing the proposed methods in the real world. In this context, eight data sets with open access have been combined to maximize the number of images used in the study. In this way, a comprehensive data set, including 3405 COVID-19, 2780 Bacterial Pneumonia, 1493 Viral Pneumonia, and 1989 Healthy images, was created. More than 90% of the images in question are taken directly from the real world.

Methods

Used data

X-ray images used in the study were obtained by combining eight different open-access data sets. Numerical information of X-ray images taken from these datasets is given in Table 1.

Table 1

Information on X-ray lung images used in the study

Source	COVID-19 Pneumonia	Other Pneumonia		Healthy
		Bacterial Pneumonia	Viral Pneumonia
Cohen et al. [28]	462	X	X	X
Wang et al. [29, 30]	35	X	X	X
Winther et al. [31, 32]	243	X	X	X
Desai et al. [33, 34]	253	X	X	X
Vayá et al. [35, 36]	2412	X	X	X
Kermany et al. [37, 38]	X	2782	1493	1583
Montgomery [39]	X	X	X	80
Shenzhen [39]	X	X	X	326
Total	3405	2782	1493	1989
	3405	4275		1989
	9669

In the study, two-class classifications were made under four different titles: COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia. As can be seen in Table 1, a total of 5303 X-ray images, including 3405 COVID-19 and 1989 Healthy, were used for COVID-19/Healthy classification. In the COVID-19 Pneumonia/Bacterial Pneumonia classification, a total of 6187 X-ray images, including 3405 COVID-19 and 2782 Bacterial Pneumonia, were used. A total of 4898 X-ray images, including 3405 COVID-19 and 1493 Bacterial Pneumonia, were used for COVID-19 Pneumonia/Viral Pneumonia classification. For COVID-19 Pneumonia/Other Pneumonia, the last classification title, a total of 7680 X-ray images, including 3405 COVID-19 and 4275 Other Pneumonia, were used. Other Pneumonia X-ray images in this classification title were obtained by combining Bacterial Pneumonia and Viral Pneumonia X-ray images. More than 90% of the X-ray images ([31-39]) used in the study were taken from the real world. When the number of COVID-19 X-ray images used in the study is compared with the number of images used in the previous studies under the Sect. 1, the number of images used in the study is more than ten times the average number of images used in previous studies.

Image pre-processing

Since images are collected from many datasets within the scope of the study, it is necessary to make the images suitable for use in the study by performing pre-processing. The images used in the study were recorded in four different formats: jpeg, jpg, png, and dicom. The sizes of these images range from 154 px × 124 px to 4064 px × 2992 px (The acronym "px" stands for pixels.). The bit depth of the images is 16, 24, 32, and 48. The images in question were first framed to cover the entire chest area, and unrelated areas were removed from the image. Then, the image was converted to gray-level format with a bit depth of 8-bit. Finally, the image dimensions have been resized to 224 px × 224 px. By applying these pre-processing processes to all X-ray images used in the study, a 9669 gray-level image matrix of 224 × 224 × 1 dimensions was obtained.

Texture analysis methods

In addition to the original images, texture feature images obtained using Local Binary Pattern (LBP) and Local Entropy (LE) methods were also used in the study. LBP is a non-parametrial feature extraction method in which the new spatial value of a pixel is compared with neighboring pixels [40]. LE displays the amount of uncertainty or randomness in an image using local histograms [41]. More detailed information about LBP and LE can be examined in the study conducted by Yaşar and Ceylan [42] on the diagnosis of COVID-19 using texture analysis methods on CT lung images.

The software used in the study was created using Matlab 2020(b). The radius value used in obtaining LBP feature images was selected as 1. The entropyfilt function was used to obtain the LE feature images. Neighborhood (nhood) parameters were used as default in operations. More detailed information about the parameters used as the default can be examined through MathWorks [43]. In the scope of the study, the images obtained using feature analysis methods were resized to 224 px × 224 px in case of a change in size. A sample selected from each of the COVID-19 Pneumonia, Bacterial Pneumonia, Viral Pneumonia, and Healthy X-ray images was used in the study. The feature images obtained by applying the relevant texture analysis methods to these images are given in Fig. 1.

Fig. 1

a COVID-19 pneumonia; b bacterial pneumonia; c viral pneumonia; d healthy X-ray image samples (Original, LBP, and LE image, respectively)

2-D and 3-D convolutional neural network (CNN)

Deep learning is an important sub-field of artificial intelligence. Generally, it generates new data and results from existing data using layers and machine learning. Convolutional Neural Network (CNN) is a deep learning approach to image processing that has been widely used recently. CNN consists of convolution layer, activation function, pooling layer, flattening, and fully connected layer components. The convolution layer is the layer on which convolution processes are performed on the image divided into as many parts as the number of filters. If the processed image is two-dimensional, convolution processes are done in two dimensions. If the images are three-dimensional, the convolution process can be done in three dimensions. In this context, the CNN architectures can be named as 2-D CNN (Two-Dimensional Convolutional Neural Networks) and 3-D CNN (Three-Dimensional Convolutional Neural Networks), respectively.

2-D versions of the CNN architectures of Mobilenetv2 [44], Resnet101 [45], and Googlenet [46] were also used within the scope of the study. However, some arrangements need to be made on these architectures to make them suitable for use in the study. The image input dimensions of Mobilenetv2, Resnet101, and Googlenet architectures are 224 × 224 × 3. In the study, the original images and the feature images obtained using LBP and LE were used directly and in combinations as the entrance image. For this reason, the input image dimensions of these architectures have been rearranged. In this context, the data entry sizes of Mobilenetv2, Resnet101, and Googlenet architectures were rearranged as 224 × 224 × 1, 224 × 224 × 2, and 224 × 224 × 3. In addition, the Fully Connected Layer output sizes of these architectures are 1000. As a two-class classification was carried out within the scope of the study, the final Fully Connected Layer output sizes of the architectures were rearranged as 2. Apart from these regulations, modification has been made on the 2-D CNN architectures.

In the scope of work, 3-D CNN architecture was also used for COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia classification. The 3-D CNN architecture utilized within the scope of the study is a modified version of the 2-D CNN architecture previously used by Yaşar and Ceylan in studies [17, 42] and translated into 3-D. In this context, the two-dimensional image input layer, convolution layer, and maximum pool layer of the 2-D CNN architecture have been replaced with three-dimensional versions. In this context, the data entry size of the 3-D CNN architecture was set to 224 × 224 × 24 × 1. Also, in 3-D CNN architecture, unlike 2-D CNN architecture, a Dropout Layer is added just before the Fully Connected Layer as a precaution against the overfitting problem. The general working structure of this 24-layer 3-D CNN architecture is shown in Fig. 2. In this CNN architecture, ReLU Layers and Batch Normalization Layers are used against the Vanishing Gradient problem. Against the overfitting problem, the Dropout Layer is used in the last part of the architecture.

Fig. 2

Architecture of 24-layer 3-D CNN architecture used in the study

Training options

The training and testing procedures used in the study were carried out according to the fivefold cross-validation principle. No initial weight transfer was performed during the training. That is, all CNN architectures are trained from the beginning with randomly assigned weights. The software used in the study was created using Matlab 2020(b). Solver for training networks used in CNN training was chosen as stochastic gradient descent with momentum (sgdm). The maximum number of epochs is selected as 30, which is accepted as the default value by the program. The main reason for this choice is to avoid the overfitting problem. In other words, instead of choosing this value as high, an early stop was made using the default value. The size of the mini-batch is set to 64. As a precaution against overfitting problems, options for data shuffling are determined as every epoch. For the study, the initial learning rate was chosen as 0.01, which is accepted as the default value by the program. All other parameters are set as default values. It is possible to examine the descriptions of the parameters and the default values through Mathworks [47].

Evaluation criteria of classification results

Confusion matrix and comparison parameters calculated using this matrix were used to evaluate the study results. In this context, confusion matrix (TP, FP, TN, and FN), sensitivity (SEN), specificity (SPE), accuracy (ACC), and F-1 Score (F-1) parameters are shown in the results tables. It is possible to calculate the comparison parameters not given in the study using a confusion matrix. Receiver-Operating Characteristic (ROC) analysis was also used within the scope of the study. The Area Under ROC Curve (AUC) values are given in the results tables. It is possible to examine the detailed explanations of the parameters in question and their mathematical calculation methods from studies [14, 16, 17].

Results

Experiments

In this study aiming at the early diagnosis of COVID-19 disease using X-ray images, automatic two-class classification has been carried out under four different titles: COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia. This study used 3405 COVID-19, 2780 Bacterial Pneumonia, 1493 Viral Pneumonia, and 1989 Healthy images obtained by combining eight different data sets with open access. A total of 5303 X-ray images, including 3405 COVID-19 and 1989 Healthy, were used for COVID-19/Healthy classification. A total of 6187 X-ray images, including 3405 COVID-19 and 2782 Bacterial Pneumonia, were used for COVID-19 Pneumonia/Bacterial Pneumonia classification. A total of 4898 X-ray images, including 3405 COVID-19 and 1493 Viral Pneumonia, were used for the COVID-19 Pneumonia/Viral Pneumonia classification. A total of 7680 X-ray images, including 3405 COVID-19 and 4,275 Other Pneumonia, were used for the COVID-19 Pneumonia/Other Pneumonia classification.

In this study, the images were first arranged and standardized. X-ray images were framed to cover the lung area completely. In this way, irrelevant areas in the X-ray images were removed from the images. Then, the images were converted to gray-level format with a bit depth of 8-bit. Finally, the images were resized to 224 px × 224 px.

Second, new feature images were obtained from the original images using texture analysis methods. In this context, two different texture analysis methods were used: Local Binary Pattern (LBP) and Local Entropy (LE). By applying these methods, new feature images with 8-bit depth and gray-level format were obtained. If the dimensions of the images obtained using feature analysis methods changed compared to the original images, the dimensions were resized to 224 px × 224 px.

Third, classification processes were made using 2-D CNN. As a 2-D CNN classifier, Mobilenetv2, Resnet101, and Googlenet whose details and changes were previously described were used. 2-D CNN input images were created using the original image and texture feature images alone and in combination. In this context, classification results were created for six different input image combinations (Original, LBP, LE, Original + LBP, Original + LE, Original + LBP + LE).

Fourth, classification processes were carried out using 3-D CNN. The input images of the 3-D CNN classifier were obtained by combining the Original, LBP, and LE images directly and in combination. The length of the third dimension of 3D input images is determined as 24. In the study, classification processes were carried out for six different input image combinations (Original × 24, LBP × 24, LE × 24, (Original + LBP) × 12, (Original + LE) × 12, (Original + LBP + LE) × 8).

Training and testing procedures were carried out according to the fivefold cross-validation. No initial weight transfer was performed during the training. All CNN architectures were trained from the beginning with randomly assigned weights. In all the experiments carried out within the scope of the study, training options as previously detailed were used. Classification times per image were also measured. Measurement times were carried out over CPU time. MATLAB 2020(b) software and Intel (R) Xeon (R) CPU E5-2680 2.7 GHz (32 CPUs) computer with a 64 GB RAM were used in the study. All CPU time in the article refers to times in seconds.

Results

COVID-19/Healthy classification results

First, for this study, a COVID-19/Healthy classification was made. A total of 5303 X-ray images were used in the classification, including 3405 COVID-19 and 1989 Healthy. Training and testing procedures were carried out using fivefold cross-validation. Classification results obtained using 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet classifier are given in Tables 2, 3, and 4, respectively. Input image sizes of 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet are 224 × 224 × 1 for Original, LBP, and LE; 224 × 224 × 2 for Original + LBP and Original + LE; and 224 × 224 × 3 for Original + LBP + LE, respectively. Table 5 contains the classification results obtained using 24-layer 3-D CNN. The dimensions of the images given as input to 24-layer 3-D CNN are 224 × 224 × 24 × 1.

Table 2

Results obtained using 2-D Mobilenetv2 for COVID-19/Healthy classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3376	29	1965	24	0.99148	0.98793	0.99017	0.99221	0.99932	1.52888
LBP	3340	65	1914	75	0.98091	0.96229	0.97405	0.97947	0.99640	1.53781
LE	3367	38	1938	51	0.98884	0.97436	0.98350	0.98696	0.99828	1.53571
Original + LBP	3372	33	1939	50	0.99031	0.97486	0.98461	0.98784	0.99884	1.60890
Original + LE	3382	23	1945	44	0.99325	0.97788	0.98758	0.99019	0.99902	1.61216
Original + LBP + LE	3376	29	1936	53	0.99148	0.97335	0.98480	0.98800	0.99871	1.67478

The highest results obtained are marked in bold

Table 3

Results obtained using 2-D Resnet101 for COVID-19/Healthy classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3378	27	1958	31	0.99207	0.98441	0.98925	0.99149	0.99878	3.29425
LBP	3370	35	1949	40	0.98972	0.97989	0.98610	0.98899	0.99882	3.30719
LE	3376	29	1950	39	0.99148	0.98039	0.98739	0.99003	0.99909	3.29880
Original + LBP	3375	30	1955	34	0.99119	0.98291	0.98813	0.99061	0.99919	3.38626
Original + LE	3369	36	1952	37	0.98943	0.98140	0.98647	0.98928	0.99912	3.37695
Original + LBP + LE	3380	25	1963	26	0.99266	0.98693	0.99055	0.99251	0.99949	3.46030

The highest results obtained are marked in bold

Table 4

Results obtained using 2-D Googlenet for COVID-19/Healthy classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3374	31	1967	22	0.99090	0.98894	0.99017	0.99221	0.99859	0.76668
LBP	3318	87	1887	102	0.97445	0.94872	0.96496	0.97231	0.99205	0.77280
LE	3371	34	1966	23	0.99001	0.98844	0.98943	0.99162	0.99922	0.77345
Original + LBP	3371	34	1930	59	0.99001	0.97034	0.98276	0.98639	0.99784	0.84699
Original + LE	3374	31	1962	27	0.99090	0.98643	0.98925	0.99148	0.99929	0.84578
Original + LBP + LE	3374	31	1943	46	0.99090	0.97687	0.98572	0.98872	0.99817	0.93245

The highest results obtained are marked in bold

Table 5

Results obtained using 24-layer 3-D CNN for COVID-19/Healthy classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
(Original) × 24	3385	20	1965	24	0.99413	0.98793	0.99184	0.99354	0.99931	4.30136
(LBP) × 24	3377	28	1957	32	0.99178	0.98391	0.98888	0.99119	0.99934	4.33525
(LE) × 24	3374	31	1955	34	0.99090	0.98291	0.98795	0.99046	0.99881	4.48620
(Original + LBP) × 12	3383	22	1966	23	0.99354	0.98844	0.99166	0.99339	0.99923	4.39649
(Original + LE) × 12	3377	28	1967	22	0.99178	0.98894	0.99073	0.99265	0.99946	4.42985
(Original + LBP + LE) × 8	3385	20	1969	20	0.99413	0.98994	0.99258	0.99413	0.99958	4.24820

The highest results obtained are marked in bold

COVID-19 Pneumonia/Bacterial Pneumonia classification results

Next, a COVID-19 Pneumonia/Bacterial Pneumonia classification was made. A total of 6187 X-ray images were used in the classification, including 3405 COVID-19 and 2782 Bacterial Pneumonia. Training and testing procedures were carried out using fivefold cross-validation. Classification results obtained using 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet classifier are given in Tables 6, 7, and 8, respectively. Input image sizes of 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet are 224 × 224 × 1 for Original, LBP, and LE; 224 × 224 × 2 for Original + LBP and Original + LE; and 224 × 224 × 3 for Original + LBP + LE, respectively. Table 9 contains the classification results obtained using 24-layer 3-D CNN. The dimensions of the images given as input to 24-layer 3-D CNN are 224 × 224 × 24 × 1.

Table 6

Results obtained using 2-D Mobilenetv2 for COVID-19 Pneumonia/Bacterial Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3388	17	2764	16	0.99501	0.99424	0.99466	0.99515	0.99934	1.52733
LBP	3376	29	2736	44	0.99148	0.98417	0.98820	0.98930	0.99903	1.52932
LE	3386	19	2751	29	0.99442	0.98957	0.99224	0.99296	0.99912	1.52560
Original + LBP	3383	22	2755	25	0.99354	0.99101	0.99240	0.99310	0.99940	1.60832
Original + LE	3392	13	2758	22	0.99618	0.99209	0.99434	0.99487	0.99891	1.60963
Original + LBP + LE	3386	19	2762	18	0.99442	0.99353	0.99402	0.99457	0.99904	1.67383

The highest results obtained are marked in bold

Table 7

Results obtained using 2-D Resnet101 for COVID-19 Pneumonia/Bacterial Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3372	33	2756	24	0.99031	0.99137	0.99078	0.99162	0.99892	3.28368
LBP	3378	27	2752	28	0.99207	0.98993	0.99111	0.99192	0.99926	3.29297
LE	3382	23	2749	31	0.99325	0.98885	0.99127	0.99208	0.99872	3.29466
Original + LBP	3382	23	2760	20	0.99325	0.99281	0.99305	0.99368	0.99934	3.35726
Original + LE	3374	31	2760	20	0.99090	0.99281	0.99175	0.99250	0.99891	3.36263
Original + LBP + LE	3385	20	2759	21	0.99413	0.99245	0.99337	0.99398	0.99925	3.43009

The highest results obtained are marked in bold

Table 8

Results obtained using 2-D Googlenet for COVID-19 Pneumonia/Bacterial Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3391	14	2752	28	0.99589	0.98993	0.99321	0.99385	0.99866	0.77360
LBP	3377	28	2743	37	0.99178	0.98669	0.98949	0.99047	0.99892	0.77290
LE	3387	18	2768	12	0.99471	0.99568	0.99515	0.99559	0.99922	0.77270
Original + LBP	3394	11	2742	38	0.99677	0.98633	0.99208	0.99283	0.99929	0.84987
Original + LE	3385	20	2750	30	0.99413	0.98921	0.99192	0.99267	0.99896	0.84982
Original + LBP + LE	3387	18	2753	27	0.99471	0.99029	0.99272	0.99340	0.99916	0.93361

The highest results obtained are marked in bold

Table 9

Results obtained using 24-layer 3-D CNN for COVID-19 Pneumonia/Bacterial Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
(Original) × 24	3381	24	2765	15	0.99295	0.99460	0.99369	0.99427	0.99909	4.13254
(LBP) × 24	3392	13	2767	13	0.99618	0.99532	0.99580	0.99618	0.99971	4.50835
(LE) × 24	3386	19	2756	24	0.99442	0.99137	0.99305	0.99369	0.99923	4.52505
(Original + LBP) × 12	3380	25	2768	12	0.99266	0.99568	0.99402	0.99456	0.99925	4.32411
(Original + LE) × 12	3384	21	2767	13	0.99383	0.99532	0.99450	0.99500	0.99942	4.32021
(Original + LBP + LE) × 8	3387	18	2766	14	0.99471	0.99496	0.99483	0.99530	0.99941	4.53055

The highest results obtained are marked in bold

COVID-19 Pneumonia/Viral Pneumonia classification results

The third classification of COVID-19 Pneumonia/Viral Pneumonia was made in this study. A total of 4898 X-ray images were used in the classification, including 3405 COVID-19 and 1493 Viral Pneumonia. Training and testing procedures were carried out using fivefold cross-validation. Classification results obtained using 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet classifier are given in Tables 10, 11, and 12, respectively. Input image sizes of 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet are 224 × 224 × 1 for Original, LBP, and LE; 224 × 224 × 2 for Original + LBP and Original + LE; and 224 × 224 × 3 for Original + LBP + LE, respectively. Table 13 contains the classification results obtained using 24-layer 3-D CNN. The dimensions of the images given as input to 24-layer 3-D CNN are 224 × 224 × 24 × 1.

Table 10

Results obtained using 2-D Mobilenetv2 for COVID-19 Pneumonia/Viral Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3391	14	1479	14	0.99589	0.99062	0.99428	0.99589	0.99942	1.92029
LBP	3382	23	1446	47	0.99325	0.96852	0.98571	0.98976	0.99807	1.87762
LE	3388	17	1472	21	0.99501	0.98593	0.99224	0.99442	0.99931	1.88044
Original + LBP	3380	25	1473	20	0.99266	0.98660	0.99081	0.99339	0.99873	1.91279
Original + LE	3391	14	1466	27	0.99589	0.98192	0.99163	0.99399	0.99954	1.93166
Original + LBP + LE	3391	14	1477	16	0.99589	0.98928	0.99388	0.99560	0.99945	1.93459

The highest results obtained are marked in bold

Table 11

Results obtained using 2-D Resnet101 for COVID-19 Pneumonia/Viral Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3371	34	1467	26	0.99001	0.98259	0.98775	0.99118	0.99867	3.32059
LBP	3391	14	1460	33	0.99589	0.97790	0.99040	0.99312	0.99852	3.34256
LE	3384	21	1463	30	0.99383	0.97991	0.98959	0.99252	0.99903	3.33412
Original + LBP	3395	10	1472	21	0.99706	0.98593	0.99367	0.99546	0.99949	3.41457
Original + LE	3379	26	1473	20	0.99236	0.98660	0.99061	0.99324	0.99890	3.42491
Original + LBP + LE	3388	17	1472	21	0.99501	0.98593	0.99224	0.99442	0.99940	3.50020

The highest results obtained are marked in bold

Table 12

Results obtained using 2-D Googlenet for COVID-19 Pneumonia/Viral Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3393	12	1470	23	0.99648	0.98459	0.99285	0.99487	0.99937	0.78366
LBP	3343	62	1434	59	0.98179	0.96048	0.97530	0.98222	0.99490	0.78533
LE	3391	14	1407	86	0.99589	0.94240	0.97958	0.98547	0.99861	0.78949
Original + LBP	3388	17	1468	25	0.99501	0.98326	0.99143	0.99384	0.99865	0.86286
Original + LE	3364	41	1478	15	0.98796	0.98995	0.98857	0.99175	0.99835	0.86187
Original + LBP + LE	3386	19	1470	23	0.99442	0.98459	0.99143	0.99384	0.99898	0.93658

The highest results obtained are marked in bold

Table 13

Results obtained using 24-layer 3-D CNN for COVID-19 Pneumonia/Viral Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
(Original) × 24	3386	19	1478	15	0.99442	0.98995	0.99306	0.99500	0.99922	4.06504
(LBP) × 24	3388	17	1466	27	0.99501	0.98192	0.99102	0.99355	0.99956	3.97470
(LE) × 24	3387	18	1472	21	0.99471	0.98593	0.99204	0.99428	0.99940	3.98068
(Original + LBP) × 12	3390	15	1473	20	0.99559	0.98660	0.99285	0.99486	0.99960	4.04915
(Original + LE) × 12	3387	18	1477	16	0.99471	0.98928	0.99306	0.99501	0.99962	4.08911
(Original + LBP + LE) × 8	3390	15	1476	17	0.99559	0.98861	0.99347	0.99530	0.99963	4.09530

The highest results obtained are marked in bold

COVID-19 Pneumonia/Other Pneumonia classification results

The last part of this study involved making the COVID-19 Pneumonia/Other Pneumonia classification. A total of 7680 X-ray images were used for classification, including 3405 COVID-19 and 4275 Other Pneumonia. Training and testing procedures were carried out using fivefold cross-validation. Classification results obtained using 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet classifier are given in Tables 14, 15, and 16, respectively. Input image sizes of 2-D Mobilenetv2, 2-D Resnet101, and 2-D Googlenet are 224 × 224 × 1 for Original, LBP, and LE; 224 × 224 × 2 for Original + LBP and Original + LE; and 224 × 224 × 3 for Original + LBP + LE, respectively. Table 17 contains the classification results obtained using 24-layer 3-D CNN. The dimensions of the images given as input to the 24-layer 3-D CNN are 224 × 224 × 24 × 1.

Table 14

Results obtained using 2-D Mobilenetv2 for COVID-19 Pneumonia/Other Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3379	26	4254	19	0.99236	0.99555	0.99414	0.99339	0.99920	1.53206
LBP	3361	44	4238	35	0.98708	0.99181	0.98971	0.98838	0.99899	1.53449
LE	3383	22	4250	23	0.99354	0.99462	0.99414	0.99339	0.99941	1.54505
Original + LBP	3382	23	4238	35	0.99325	0.99181	0.99245	0.99150	0.99919	1.61698
Original + LE	3390	15	4258	15	0.99559	0.99649	0.99609	0.99559	0.99953	1.61581
Original + LBP + LE	3379	26	4242	31	0.99236	0.99275	0.99258	0.99164	0.99937	1.68222

The highest results obtained are marked in bold

Table 15

Results obtained using 2-D Resnet101 for COVID-19 Pneumonia/Other Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3371	34	4248	25	0.99001	0.99415	0.99232	0.99132	0.99895	3.29616
LBP	3385	20	4226	47	0.99413	0.98900	0.99127	0.99020	0.99882	3.29504
LE	3379	26	4241	32	0.99236	0.99251	0.99245	0.99149	0.99924	3.29188
Original + LBP	3386	19	4251	22	0.99442	0.99485	0.99466	0.99398	0.99959	3.37402
Original + LE	3369	36	4229	44	0.98943	0.98970	0.98958	0.98827	0.99886	3.36625
Original + LBP + LE	3377	28	4240	33	0.99178	0.99228	0.99206	0.99105	0.99935	3.43928

The highest results obtained are marked in bold

Table 16

Results obtained using 2-D Googlenet for COVID-19 Pneumonia/Other Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
Original	3362	43	4262	11	0.98737	0.99743	0.99297	0.99203	0.99922	0.77573
LBP	3013	392	4235	38	0.88488	0.99111	0.94400	0.93340	0.99150	0.77669
LE	3368	37	4233	40	0.98913	0.99064	0.98997	0.98870	0.99843	0.77822
Original + LBP	3374	31	4256	17	0.99090	0.99602	0.99375	0.99294	0.99957	0.85580
Original + LE	3364	41	4240	33	0.98796	0.99228	0.99036	0.98912	0.99911	0.85452
Original + LBP + LE	3386	19	4251	22	0.99442	0.99485	0.99466	0.99398	0.99942	0.92986

The highest results obtained are marked in bold

Table 17

Results obtained using 24-layer 3-D CNN for COVID-19 Pneumonia/Other Pneumonia classification

Method	TP	FN	TN	FP	SEN	SPE	ACC	F-1	AUC	CPU time
(Original) × 24	3373	32	4255	18	0.99060	0.99579	0.99349	0.99264	0.99938	4.83264
(LBP) × 24	3390	15	4259	14	0.99559	0.99672	0.99622	0.99574	0.99976	4.98608
(LE) × 24	3376	29	4244	29	0.99148	0.99321	0.99245	0.99148	0.99943	5.15694
(Original + LBP) × 12	3387	18	4258	15	0.99471	0.99649	0.99570	0.99515	0.99967	4.66094
(Original + LE) × 12	3392	13	4247	26	0.99618	0.99392	0.99492	0.99428	0.99971	4.93204
(Original + LBP + LE) × 8	3391	14	4254	19	0.99589	0.99555	0.99570	0.99516	0.99981	5.08685

The highest results obtained are marked in bold

Discussion

The results of the experiments performed for the classification of COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia are shared between Table 2 and Table 17. In this section, the general evaluation of the results obtained has been made. For the COVID-19/Healthy classification, the comparison of the best results obtained using the feature images directly or in combination with the original image is summarized in Table 18.

Table 18

Comparison of the best results obtained in experiments (COVID-19/Healthy classification)

CNN type	Input type	SEN	SPE	ACC	F-1	AUC	CPU time
2-D Mobilenetv2	Original + LE	0.99325	0.97788	0.98758	0.99019	0.99902	1.61216
2-D Mobilenetv2	Original	0.99148	0.98793	0.99017	0.99221	0.99932	1.52888
2-D Resnet101	Original + LBP + LE	0.99266	0.98693	0.99055	0.99251	0.99949	3.46030
2-D Googlenet	Original	0.99090	0.98894	0.99017	0.99221	0.99859	0.76668
2-D Googlenet	Original + LBP + LE	0.99090	0.97687	0.98572	0.98872	0.99817	0.93245
2-D Googlenet	Original + LE	0.99090	0.98643	0.98925	0.99148	0.99929	0.84578
24-layer 3-D CNN	(Original) × 24	0.99413	0.98793	0.99184	0.99354	0.99931	4.30136
24-layer 3-D CNN	(Original + LBP + LE) × 8	0.99413	0.98994	0.99258	0.99413	0.99958	4.24820

The highest results obtained are marked in bold

Table 18 shows that the most successful results are obtained using 24-layer 3-D CNN. 2-D Resnet101, 2-D Mobilenetv2, and 2-D Googlenet architectures follow the 24-layer 3-D CNN architecture in terms of success, respectively. Another important issue is the input image combinations that ensure the most successful results. In this context, the use of Original + LE and Original + LBP + LE combinations, as well as the direct use of the original images, provides successful results. When comparing run times, the costliest working time belongs to the 24-layer 3-D CNN architecture, and the least costly working time belongs to the 2-D Googlenet architecture.

Similar comparisons to that performed in Table 18 are provided in Table 19 for the COVID-19 Pneumonia/Bacterial Pneumonia classification, Table 20 for the COVID-19 Pneumonia/Viral Pneumonia classification, and Table 21 for the COVID-19 Pneumonia/Other Pneumonia classification, respectively.

Table 19

Comparison of the best results obtained in experiments (COVID-19 Pneumonia/Bacterial Pneumonia classification)

CNN type	Input type	SEN	SPE	ACC	F-1	AUC	CPU time
2-D Mobilenetv2	Original + LE	0.99618	0.99209	0.99434	0.99487	0.99891	1.60963
2-D Mobilenetv2	Original	0.99501	0.99424	0.99466	0.99515	0.99934	1.52733
2-D Mobilenetv2	Original + LBP	0.99354	0.99101	0.99240	0.99310	0.99940	1.60832
2-D Resnet101	Original + LBP + LE	0.99413	0.99245	0.99337	0.99398	0.99925	3.43009
2-D Resnet101	Original + LE	0.99090	0.99281	0.99175	0.99250	0.99891	3.36263
2-D Resnet101	Original + LBP	0.99325	0.99281	0.99305	0.99368	0.99934	3.35726
2-D Googlenet	Original + LBP	0.99677	0.98633	0.99208	0.99283	0.99929	0.84987
2-D Googlenet	LE	0.99471	0.99568	0.99515	0.99559	0.99922	0.77270
24-layer 3-D CNN	(LBP) × 24	0.99618	0.99532	0.99580	0.99618	0.99971	4.50835
24-layer 3-D CNN	(Original + LBP) × 12	0.99266	0.99568	0.99402	0.99456	0.99925	4.32411

The highest results obtained are marked in bold

Table 20

Comparison of the best results obtained in experiments (COVID-19 Pneumonia/Viral Pneumonia classification)

CNN type	Input type	SEN	SPE	ACC	F-1	AUC	CPU time
2-D Mobilenetv2	Original	0.99589	0.99062	0.99428	0.99589	0.99942	1.92029
2-D Mobilenetv2	Original + LBP + LE	0.99589	0.98928	0.99388	0.99560	0.99945	1.93459
2-D Mobilenetv2	Original + LE	0.99589	0.98192	0.99163	0.99399	0.99954	1.93166
2-D Resnet101	Original + LBP	0.99706	0.98593	0.99367	0.99546	0.99949	3.41457
2-D Resnet101	Original + LE	0.99236	0.98660	0.99061	0.99324	0.99890	3.42491
2-D Googlenet	Original	0.99648	0.98459	0.99285	0.99487	0.99937	0.78366
2-D Googlenet	Original + LE	0.98796	0.98995	0.98857	0.99175	0.99835	0.86187
24-layer 3-D CNN	(Original + LBP) × 12	0.99559	0.98660	0.99285	0.99486	0.99960	4.04915
24-layer 3-D CNN	(Original + LBP + LE) × 8	0.99559	0.98861	0.99347	0.99530	0.99963	4.09530
24-layer 3-D CNN	(Original) × 24	0.99442	0.98995	0.99306	0.99500	0.99922	4.06504

The highest results obtained are marked in bold

Table 21

Comparison of the best results obtained in experiments (COVID-19 Pneumonia/Other Pneumonia classification)

CNN type	Input type	SEN	SPE	ACC	F-1	AUC	CPU time
2D-Mobilenetv2	Original + LE	0.99559	0.99649	0.99609	0.99559	0.99953	1.61581
2D-Resnet101	Original + LBP	0.99442	0.99485	0.99466	0.99398	0.99959	3.37402
2D-Googlenet	Original + LBP + LE	0.99442	0.99485	0.99466	0.99398	0.99942	0.92986
2D-Googlenet	Original	0.98737	0.99743	0.99297	0.99203	0.99922	0.77573
2D-Googlenet	Original + LBP	0.99090	0.99602	0.99375	0.99294	0.99957	0.85580
24-layers 3D-CNN	(Original + LE) × 12	0.99618	0.99392	0.99492	0.99428	0.99971	4.93204
24-layers 3D-CNN	(LBP) × 24	0.99559	0.99672	0.99622	0.99574	0.99976	4.98608
24-layers 3D-CNN	(Original + LBP + LE) × 8	0.99589	0.99555	0.99570	0.99516	0.99981	5.08685

The highest results obtained are marked in bold

When Tables 19, 20, 21 are examined, it is seen that the CNN architectural success order given for Table 18 is valid, although there are slight differences. In addition, it is understood that the most successful results were obtained using Original + LBP, Original + LE, Original + LBP + LE input image combinations, as well as using the original images directly. When comparing the run times, it is seen that 24-layer 3-D CNN, 2-D Resnet101, 2-D Mobilenetv2, and 2-D Googlenet are in order of costliest to the least costly.

The comparison of the results obtained for the classification of COVID-19/Healthy within the scope of the study with the previous studies in the literature is given in Table 22. The comparison of the results obtained for the classification of COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia with previous studies in the literature can be seen in Table 23.

Table 22

Comparison of the results with previous studies for COVID-19/Healthy classification

Study	SEN	SPE	ACC	F-1	AUC
Tuncer et al. [11]	0.8149–1.0000	0.9380–1.0000	0.9049–0.9955	X	X
Panwar et al. [12]	0.9762	0.7857	0.881	X	X
Ozturk et al. [13]	0.9513	0.953	0.9808	0.9651	X
Mohammed et al. [14]	0.706–0.974	0.557–1.000	0.620–0.987	0.555–0.987	0.800–0.988
Toraman et al. [15]	0.28–0.9742	0.8095–0.98	0.4914–0.9724	0.55–0.9724	X
Khan et al. [16]	0.993	0.986	0.990	0.985	X
Yasar and Ceylan [17]	0.9947	1.0000	0.9906	0.9881	0.9997
Waheed et al. [18]	0.69–0.90	0.95–0.97	0.85–0.95	X	X
Duran-Lopez et al. [19]	0.9253	0.9633	0.9443	0.9314	0.988
Vaid et al. [20]	0.9863	0.9166	0.9633	0.9729	X
Benbrahim et al. [21]	0.9803–0.9811	X	0.9803–0.9901	0.9803–0.9901	X
Loey et al. [22]	1.0000	1.0000	1.0000	X	X
Minaee et al. [23]	0.98	0.751–0.929	X	X	X
Elaziz et al. [24]	0.9875–0.9891	X	0.9609–0.9809	X	X
Martínez et al. [25]	0.97	X	0.97	0.97	X
Mahmud et al. [26]	0.978	0.947	0.974	0.971	0.969
24-layers 3D-CNN/(Original) × 24	0.99413	0.98793	0.99184	0.99354	0.99931
24-layers 3D-CNN/(Original + LBP + LE) × 8	0.99413	0.98994	0.99258	0.99413	0.99958

The highest results obtained are marked in bold

Table 23

Comparison of the results with previous studies for COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral pneumonia, and COVID-19 Pneumonia/Other Pneumonia classification

Study	Classes	SEN	SPE	ACC	F-1	AUC
Mahmud et al. [26]	COVID-19/Bacterial Pan	0.944	0.933	0.947	0.939	0.951
Mahmud et al. [26]	COVID-19/Viral Pn	0.874	0.855	0.873	0.878	0.921
Horry et al. [27]	COVID-19/Other Pn	0.86–0.89	X	X	0.86–0.89	X
2-D Googlenet/Original + LBP	COVID-19/Bacterial Pn	0.99677	0.98633	0.99208	0.99283	0.99929
2-D Googlenet/LE	COVID-19/Bacterial Pn	0.99471	0.99568	0.99515	0.99559	0.99922
24-layers 3D-CNN/(Original + LBP) × 12	COVID-19/Bacterial Pn	0.99266	0.99568	0.99402	0.99456	0.99925
24-layers 3D-CNN/(LBP) × 24	COVID-19/Bacterial Pn	0.99618	0.99532	0.99580	0.99618	0.99971
2-D Googlenet/Original	COVID-19/Viral Pn	0.99648	0.98459	0.99285	0.99487	0.99937
2-D Googlenet/ Original + LE	COVID-19/Viral Pn	0.98796	0.98995	0.98857	0.99175	0.99835
24-layers 3D-CNN/(Original) × 24	COVID-19/Viral Pn	0.99442	0.98995	0.99306	0.99500	0.99922
24-layers 3D-CNN/(Original + LBP + LE) × 8	COVID-19/Viral Pn	0.99559	0.98861	0.99347	0.99530	0.99963
24-layer 3D CNN/((Original + LE) × 12	COVID-19/Other Pn	0.99618	0.99392	0.99492	0.99428	0.99971
2-D Googlenet/Original	COVID-19/Other Pn	0.98737	0.99743	0.99297	0.99203	0.99922
24-layer 3D CNN/((LBP) × 24	COVID-19/Other Pn	0.99559	0.99672	0.99622	0.99574	0.99976
24-layer 3D CNN/((Original + LBP + LE) × 8	COVID-19/Other Pn	0.99589	0.99555	0.99570	0.99516	0.99981

The highest results obtained are marked in bold

Conclusion

Detailed results have been obtained within the scope of the study, which recommends the automatic classification of X-ray chest images for early diagnosis of COVID-19. In this study, automatic two-class classification was carried out under four different titles: COVID-19/Healthy, COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia. The texture analysis methods Local Binary Pattern, Local Entropy (LE), 2-D CNN, and 3-D CNN classifier were used in the study.

When the summary results in Table 18 and Table 21 are examined, it is seen that the diversification of CNN input images with texture analysis methods has a positive effect on the study results. In addition, these improvements in the study results occurred for all CNN classifiers used in the study. These improvements are due to the enrichment of the entrance images using the texture feature images in addition to the original images. Because the input images contain more information, the success achieved in classification processes is increasing. The 2-D CNN architectures used in the study are generally parallel or hybrid (a combination of series and parallel), not series. This situation prevents an increase in working times at the same rate despite the increase in the third color space dimensions of the input images. In other words, the enriching use of the entrance images has a very limited effect on working times.

If making a general comparison between texture analysis methods within the scope of the study, using the original images in combination with the images obtained using LE and LBP (Original + LBP, Original + LE, Original + LBP + LE) is more effective in terms of increasing the results of the study. When the CNN classifiers are compared, it is understood that the 24-layer 3-D CNN classifier is more successful than the other 2-D CNN classifiers. However, due to the three-dimensional convolution processes and the increase in input image sizes, the result generation time of the 24-layer 3-D CNN classifier is the slowest among CNN classifiers.

Table 22, which compares the results obtained within the study for the COVID-19/Healthy classification and the results obtained in the previous studies, shows that important results were obtained in this study. The sensitivity parameter obtained within the study is higher than 14 of the 16 studies in which this parameter was calculated. Also, the specificity parameter is better than 9 of the 13 studies. When a comparison is made between the accuracy parameters, it is seen that the result obtained within the study is higher than 13 of the 15 studies in which this parameter was calculated. In addition, the F-1 Score parameter is higher than all ten studies in which this parameter was calculated, and the AUC parameter is higher than all four studies in which this parameter was calculated.

In this study, the results obtained for the COVID-19/Healthy classification fell behind the results obtained in the studies carried out by Tuncer et al. [11], Loey et al. [22], and Mohammed et al. [14]. However, to make a more detailed comparison, the number of images used in the studies and the methods used in training and testing processes should also be considered. The number of COVID-19 X-ray images used in the study was 40 times the number of images used in the study by Tuncer et al. [11], 50 times the number of images used in the study by Loey et al. [22], and 135 times the number of images used in the study by Mohammed et al. [14]. The fact that the number of COVID-19 X-ray images used in the study is quite high compared to other studies indicates that the results obtained are more stable. In addition, the CNN architectures used in the study carried out by Loey et al. [22] were also used as a classifier in this study. Another issue is that k-fold cross-validation was performed within the study. However, the training–testing procedure used in the study by Loey et al. [22] is different from the k-fold cross-validation.

Table 23 compares the results obtained within the study for the COVID-19 Pneumonia/Bacterial Pneumonia, COVID-19 Pneumonia/Viral Pneumonia, and COVID-19 Pneumonia/Other Pneumonia classification, and the results obtained in the previous studies. The sensitivity, specificity, accuracy, F-1, and AUC results obtained within the scope of the study appear to be higher than the results obtained in the previous studies. It is important to consider that the number of COVID-19 X-ray images used in the study for these classification titles is quite high compared to other studies.

In the studies to be carried out after this stage, the aim is to create more 3-D CNN architectures and to perform classification experiments. Another important work topic is to develop new Complex 2-D CNN and Complex 3-D CNN architectures and test their performance. Testing tissue analysis methods other than the tissue analysis method used within this study will help increase the success of the study. These further studies also aim to reveal results for multi-class classification (three-class and four-class).