Automatic classification of severity of COVID-19 patients using texture feature and random forest based on computed tomography images.

Severity assessment of the novel Coronavirus (COVID-19) using chest computed tomography (CT) scan is crucial for the effective administration of the right therapeutic drugs and also for monitoring the progression of the disease. However, determining the severity of COVID-19 needs a highly expert radiologist by visual assessment, which is time-consuming, boring, and subjective. This article introduces an advanced machine learning tool to determine the severity of COVID-19 to mild, moderate, and severe from the lung CT images. We have used a set of quantitative first- and second-order statistical texture features from each image. The first-order texture features extracted from the image histogram are variance, skewness, and kurtosis. The second-order texture features extraction methods are gray-level co-occurrence matrix, gray-level run length matrix, and gray-level size zone matrix. Finally, using the extracted features, CT images of each person are classified using random forest (RF) as an ensemble method based on majority voting of the decision trees outputs to four classes. We have used a dataset of CT scans labeled as being normal (231), mild (563), moderate (120), and severe (42) determined by expert radiologists. The experimental results indicate the combination of all feature extraction methods, and RF achieves the highest result compared with the other strategies in detecting the four classes of severity of COVID-19 from CT images with an accuracy of 90.95%. This proposed system can work well and can be used as an assistant diagnostic tool for quantification of lung involvement of COVID-19 to monitor the progression of the disease.

INTRODUCTION

In December 2019, the novel Coronavirus (COVID‐19) was reported in Wuhan, China. A month later, it was identified in many other countries around the world, and the World Health Organization (WHO) announced that the COVID‐19 pandemic as a public health emergency. COVID‐19 is an infection that is highly contagious and has spread rapidly worldwide. This is not only due to its fast person‐to‐person spread but also because most infected people are not immune to it. , The diagnostic tests for COVID‐19 are mostly based on reverse transcription‐polymerase chain reaction (RT‐PCR). However, PCR results often require more than 5–6 h and also have a false negative. Moreover, the severity of the disease is not detectable by this method. For these reasons, a rapid and accurate diagnostic method is required to isolate and treat the patients as soon as possible.

Chest computed tomography (CT) scan is another way to diagnose COVID‐19, which is more efficient and faster test. Furthermore, lung CT images contain important information as the severity of disease with COVID‐19 based on lung involvement. According to the guidelines, lung CT findings of COVID‐19 are ground glass opacity (GGO), crazy‐paving pattern, and subsequent consolidation. , Severity assessment is crucial for the effective administration of the right therapeutic drugs and choosing the best cure for them and also useful for following the changes of the patient's condition. , , Follow‐up scans every 3–5 days for patients with COVID‐19 are often recommended for disease progression. So, from the beginning of this epidemic until now, processing of lung CT images has played an important role in clinical discovery of suspected cases, a suitable tool for treatment review , and monitoring the disease. , , , The severity of COVID‐19 can be done based on the assessment of CT images by a highly expert radiologist, visually. But visual evolution is tedious and subjective. So, quantitative analysis of CT images using an advanced machine learning tool as an automatic, reliable, and objective estimation of the disease is needed for solving this problem during the COVID‐19 pandemic.

Recently, there have been some research studies for automatic detection of COVID‐19 disease based on lung CT images using artificial intelligence. Chen et al. used UNet++, a novel architecture for medical image segmentation, to extract infected areas in CT images for identification of COVID‐19 pneumonia. The other researchers have used convolutional neural network (CNN) and deep transfer learning algorithms for detecting COVID‐19 cases. , , , Finally, a feature extraction method from CT images using the second‐order statistical texture features such as gray‐level co‐occurrence matrix (GLCM) and local directional pattern (LDP), as well as discrete wavelet transform (DWT), is proposed. Then, researchers used support vector machines (SVM) method to classify CT images to detect the COVID‐19. Zhang et al. used AlexNet (a pretrained deep learning network) to diagnose COVID‐19 from chest CT. Then, it was enhanced by adding batch normalization to reduce internal covariance shift and accelerate the training and also replacing the fully connected layer in AlexNet with three different classifiers. But the above studies are only about automatic detection of COVID‐19 on CT images and do not consider the severity of it.

Few studies worked on the CT images to determine the severity of patients based on lung involvement. In one recent study, changes in chest CT findings (GGO, crazy‐paving pattern, and consolidation) of 21 patients associated with COVID‐19 from initial diagnosis until patient recovery are determined. Based on this study, most patients who recovered from COVID‐19 pneumonia showed the greatest severity of lung disease on CT at approximately 10 days after the initial onset of symptoms. In another study, U‐Net deep learning method was used for segmentation of CT lung opacification of the whole lung and then classified the CT images of 126 patients with COVID‐19 into mild, moderate, severe, and critical cases. Finally, image segmentation system based on deep learning methods has also been developed to automatically quantify infection regions of chest CT scans of COVID‐19 patients and identify lesion percentage cover and mean lesion density to evaluate 3‐class severity in 44 patients. However, these few methods need image segmentation of lesion percentage of lung involvement in COVID‐19 patients, which is a problemist task.

The aim of this article is to propose a new automatic system for diagnosing the severity of COVID‐19 from the lung CT images to four classes as normal, mild, moderate, and severe without any need to image segmentation based on a combination of first and second statistical features and random forest (RF) classifier as advanced machine learning method. This proposed severity assessment system can help physicians to observe the changes in the patients’ condition and take better decisions to cure and finally control infections for COVID‐19 patients.

MATERIALS AND METHODS

Dataset

Lung CT images in the sagittal views from people suspected of having COVID‐19 were used in this study from free open access datasets provided by municipal hospitals in Moscow, Russia. These images were obtained between March 1, 2020, and April 25, 2020. Noncontrast standard dose CT scan data were acquired using Toshiba Aquilion 64 machine with number of slices 64, slice thickness 1 mm. Table height and alignment are adjusted, and gantry of CT machine is centered at the thorax of patients. A total of 956 patients (males—42%, females—56%, other/unknown—2%) were included with age from 18 to 97 years, median—47 years, max: 97 years old. Clinical information of patients is shown in Table 1.

TABLE 1

Clinical information of 956 patients (males—42%, females—56%, other/unknown—2%) with age from 18 to 97 years, median: 47 years, max: 97 years old (respiratory rate [RR], temperature [t], oxygen saturation [spo2])

Severity	Clinical data
Class 1: normal range (231 cases)	Normal and no pulmonary parenchymal involvement
Class 2: mild (563 cases)	t < 38°c, RR < 20/min, spo2 > 95, percent of pulmonary parenchymal involvement = <25%
Class 3: Moderate (120 cases)	t < 38.5°c, RR 20–30/min, spo2: 95, percent of pulmonary parenchymal involvement = 25%–50%
Class 4: Severe and critical (42 cases)	(t > 38.5°c, RR > 30/min, spo2 < 95 (signs of shock, multiple organ failure, respiratory failure.) percent of pulmonary parenchymal involvement > = 75%

CT images are categorized into five classes including normal lung tissue without symptoms of COVID‐19 (231 cases) and COVID‐19‐related findings in four different levels depending on the percent of pulmonary parenchymal involvement from mild (563 cases), moderate (120 cases), severe (40 cases), and critical (2 cases) by highly experienced radiologist. In mild and moderate stages, several GGO are obtained, and involvement of lung parenchyma in mild is less than 25% and in moderate is between 25% and 50%. In the severe stage, GGO and regions of consolidation are obtained, and involvement of lung parenchyma is between 50% and 75%. Finally, in the critical stage, diffuse GGO and consolidation as well as reticular changes in lungs are obtained. Involvement of lung parenchyma exceeds 75%. Due to the limited amount of data in the severe and critical stages, we considered the images of these two stages to be one stage as named severe stage. So, in this study, CT images are categorized into four classes including normal lung tissue (231 cases) and COVID‐19‐related findings in three levels of severity from mild (563 cases), moderate (120 cases), and severe (42 cases). Figure 1 shows some CT slices of four normal, mild, moderate, and severe COVID‐19 cases in our dataset.

FIGURE 1

Lung computed tomography (CT) slices of normal, mild, moderate, and severe COVID‐19 cases

Preprocessing

In lung CT imaging, a set of images approximately between 20 and 30 slices is taken for each patient from the upper part of the chest to the bottom part. In order to find the best images that contain lung organ completely from the collection of images, we removed the upper and bottom slices of the chest CT images during the preprocessing stage. So, the images that clearly contain lung volume are created. Then, from each image, 20% on both sides of the vertical part and 5% of the horizontal part are cropped, automatically. It is because of the anatomical structure of the human body and the slices of CT scan images that are oval shaped. Finally, histogram equalization is applied in each image for equalizing the gray level of image lightness.

Feature extraction methods

In order to classify the patients into four classes, a set of quantitative features must be extracted from the CT images. We have used the first‐ and second‐order statistical texture features from each image. These statistical features provide information about the gray level of CT images and also about the spatial arrangement of intensities. Variance, skewness, and kurtosis as the first‐order statistical features are extracted based on the image histogram of CT images and we named them global features in the results. Variance shows the variations in the intensity of the image histogram. Skewness indicates the relative level of the slope of the curve of image histogram, and kurtosis indicates the level of sharpness of the curve of the image histogram. The second‐order statistical texture features extraction methods from each image used in this study are GLCM, gray level run length matrix (GLRLM), and gray‐level size zone matrix (GLSZM). GLCM feature extraction methods in this study are autocorrelation, contrast, correlation1, correlation2, cluster prominence, cluster shade, dissimilarity, energy, entropy, homogeneity, maximum probability, sum of squares, variance, sum average, sum variance, sum entropy, difference variance, difference entropy, information measure of correlation1, information measure of correlation2, inverse difference, inverse difference normalized, and normalized inverse difference moment. We applied these 23 GLCM features in four principal degrees (horizontal [0°], anti‐diagonal [45°], vertical [90°], and diagonal [135°]) with one step. Then, GLCM features at four directions are averaged. GLRLM is another widely used method to extract high‐level second statistical texture features for medical images. It calculates the number of pair pixels of gray‐level value and its length of runs in a considered region of interest. This method is also applied in four principal direction (horizontal [0°], anti‐diagonal [45°], vertical [90°], and diagonal [135°]). GLSZM is a developed version of the GLRLM method, which is composed of large areas of the same intensity, and not of small groups of pixels or segments in any given direction. Consequently, 28 features (3 first‐order statistical features and 25 second‐order features) have been measured in this study for each image in the MATLAB Image Processing Toolbox. Then, for each person, these mentioned features are extracted from all CT images. Finally, the sum of these features from all CT images is considered as the feature sets of that person.

Classification methods

RF classifier as a supervised machine learning algorithm is applied to classify CT images of each person into four classes as normal, mild, moderate, and severe COVID‐19. RF algorithm is one of the most successful classifiers for multiclass problems. , , It is an ensemble classifier that consists of a set of decision trees, and the final decision of RF is the majority voting among all of the trees to make it more accurate and stable. A combination of different learning models leads to a better model to increase the overall results. In the training phase, each decision tree is constructed using a subset of the training sample, which is drawn from the total training sample using the bootstrap sampling method. So, each decision tree is fed by only a small number of randomly selected features and also dataset, randomly. This algorithm looks for the best selected features for different decision trees. Then, decision trees constructed according to each subset are typically grown and the classification results are obtained. Finally, the decision of RF as the optimal results is determined using majority voting among the result of different trees, which reacts independently. For comparison, linear discriminant analysis (LDA) and k‐nearest neighbor (KNN) methods are also used to classify CT images into four classes. LDA is a supervised algorithm that classifies dataset into a lower‐dimensional space with good class separability and maximizes the separation between two classes. In the KNN classification, which is very simple, highly efficient, and effective, test data are classified by a majority vote of k neighboring classes.

RESULT

In this study, a new automatic methodology is performed to classify chest CT images of 956 people into four classes as normal, mild, moderate, and severe COVID‐19 depending on the percent of pulmonary parenchymal involvement. Figure 2 shows the block diagram of our method. All processing was performed using MATLAB R2019b on a 4.5 GHz quad‐core computer with NVIDIA GeForce RTX 1660Ti graphics processing unit (GPU). In the preprocessing step, CT slices that contain lung organs from all chest CT slices are taken. Then, from each image, 20% on both sides of the vertical part and 5% of the horizontal part of the image are cropped. Finally, histogram equalization is applied in each image. The next step is to extract the features from the CT images. We have used the first‐ and second‐order statistical texture features from each image. The first‐order statistical texture features from the image histogram named variance, skewness, and kurtosis are extracted. The second‐order statistical texture features extraction methods are GLCM (23 features), GLRLM, and GLSZM. Consequently, for each person, 28 features (3 first‐order statistical features and 25 second‐order features) are extracted from all CT images (approximately between 20 and 30). The sum of these features from all CT images is considered as the feature sets of that person. Finally, using the extracted features, CT images of each person are classified into four classes as normal, mild, moderate, and severe COVID‐19 using RF classifier. Table 2 shows the values of classification accuracy by different feature extraction methods. We compared RF classifier with two other common classifiers: LDA and KNN, to classify CT images into four classes. The numbers of neighbors for KNN classifier are optimized by applying the trial and error method and set to 8 to obtain the best results. For assessment of the classifiers, the different k‐fold cross‐validation methods were used and the best results for eightfold were obtained and reported in Table 2. In the appendix section (Table A1), results from fivefold and 10‐fold are compared with our proposed method (eightfold).

FIGURE 2

Classification process for computed tomography (CT) images to normal, mild, moderate, and severe COVID‐19

TABLE 2

The results of mean classification accuracy for eightfold cross‐validation

Classification with feature extraction method	Normal	Mild	Moderate	Severe	Four classes
LDA with GLCM	89.2	86.64	28.57	80	79.65
LDA with GLRLMS	93.23	92.85	33.84	100	85.81
LDA with GLSZM	84.87	86.96	33.77	80	79.50
LDA with Global	86.16	87.14	26.28	80	78.92
KNN with GLCM	86.2	85.76	28.87	80	78.45
KNN with GLRLMS	93.19	93.46	38.89	100	86.80
KNN with GLSZM	80.14	85.38	26.66	80	76.47
KNN with Global	86.54	84.64	26.32	80	77.55
RF with GLCM	86.33	87.88	26.22	80	79.39
RF with GLRLMS	95.1	98.2	28.57	100	88.74
RF with GLSZM	79.31	81.79	24.32	80	73.86
RF with Global	89.95	85.97	28.47	80	79.44
LDA with GLCM + GLRLMS + GLSZM + Global	93.1	97.14	33.33	80	87.23
KNN with GLCM + GLRLMS + GLSZM + Global	95.49	92.98	26.98	80	84.73
RF with GLCM + GLRLMS + GLSZM + Global	96.6	98.88	40	100	90.95

Abbreviations: KNN, k‐nearest neighbor; LDA, linear discriminant analysis; GLCM, gray‐level co‐occurrence matrix; RF, random forest.

Note: The significance of bold values are the best‐obtained results.

TABLE A1

The results of mean classification accuracy for fivefold, eightfold, and 10‐fold cross‐validation

Classification with feature extraction method	Fivefold	Eightfold	10‐fold
LDA with GLCM	61.34	79.65	79.28
LDA with GLRLMS	68.07	85.81	85.23
LDA with GLSZM	58.82	79.50	79.45
LDA with Global	57.48	78.92	79.03
KNN with GLCM	62.86	78.45	78.73
KNN with GLRLMS	67.65	86.80	86.48
KNN with GLSZM	61.18	76.47	76.28
KNN with Global	61.93	77.55	77.36
RF with GLCM	68.45	79.39	79.12
RF with GLRLMS	71.90	88.74	88.68
RF with GLSZM	60.33	73.86	73.80
RF with Global	63.53	79.44	79.45
LDA with GLCM + GLRLMS + GLSZM + Global	70.98	87.23	87.01
KNN with GLCM + GLRLMS + GLSZM + Global	69.14	84.73	84.62
RF with GLCM + GLRLMS + GLSZM + Global	72.98	90.95	90.01

Abbreviations: KNN, k‐nearest neighbor; LDA, linear discriminant analysis; GLCM, gray‐level co‐occurrence matrix; RF, random forest.

Note: The significance of bold values are the best‐obtained results.

It is shown RF with GLRLMS feature extraction method yields a classification accuracy of 88.74% in detecting the four classes of severity of COVID‐19. Clearly, it is demonstrated that RF with GLRLMS achieves the highest result in recognition of CT images compared with the other mentioned methods. To improve the classification results, we combined all feature extraction methods. The best result for combination of all features was also obtained with the RF classifier with an accuracy of 90.95%. The p value of Wilcoxon rank‐sum test for the best results and RF with GLRLMS is 0.008, which demonstrates the statistical significance of our proposed approach (p < 0.05). So, the RF classifiers performed more powerfully than the others. It should be noted that the number of trees in RF is a structural parameter and needs to be tuned in an optimization process. Therefore, the classification error for different number of trees (10–70 trees) is calculated. As the number of trees increases, more decision units participate in voting. The optimum number is where adding more trees does not significantly improve the result. The best value for the tree number is set to 40 in order to achieve the best performance. The results of these computations for selecting the best number of trees are demonstrated in Figure 3. Finally, a comparison of estimation of labels provided by the proposed method (RF classifier with all combination of features) against those assigned by the reference one (highly experienced radiologist) is achieved and presented in Table 3. It shows our classifier had acceptable accuracy with 96.6% in normal class, 98.6% in mild class, and 100% in severe class, but our bad result was accurate up to 60% in moderate COVID‐19.

FIGURE 3

The results of selecting the best number of trees in RF classifier with all combinations of features. The optimum feature number is 40

TABLE 3

Comparison of estimation of labels provided by the proposed method (RF classifier with all combination of features) against those assigned by the reference one (highly experienced radiologist)

COVID‐19		Estimated labels by the proposed method
		Normal	Mild	Moderate	Severe
Reference	Normal	28	1	0	0
	Mild	1	69	0	0
	Moderate	0	9	6	0
	Severe	0	0	0	5

DISCUSSION

In this study, we have proposed an automatic method for diagnosis of severity of COVID‐19 to normal, mild, moderate, and severe based on lung involvement percentage from all lung CT images of a person. We have demonstrated that the proposed method based on a combination of first and second statistical texture features along with an RF classifier has higher results with an accuracy of 90.95%. So, it can help physicians to take better decisions for monitoring the patient and choose the best cure for them based on diagnosed level of a patient.

Our problem in the proposed method is to separate CT images of mild and moderate classes. Our classifier separated class normal and severe very well, but moderate and mild classes are mistaken for each other. It is because of the similarity of finding CT patterns in these two classes. CT finding for patients with mild and moderate levels of COVID‐19 is GGO with different involvement of lung parenchyma (less than 25% in mild and between 25 and 50 in moderate). So, the patterns of these images are really similar, and it was too difficult for a classifier to separate them exactly. Another reason could be a smaller number of datasets in moderate level in comparison with mild level, and consequently moderate classes are detected to mild class wrongly (Table 2). So, because of the similarity of pattern and the low number of datasets in the moderate level, our classifier is unable to separate these two classes very well.

Table 4 compares the results of our work with other studies in the classification of severity of COVID‐19 patients from CT images. As it is observed, the performance achieved in this study is promising. Compared with other research studies, this work has some advantages and shortcomings. Our method has used a combination of the first‐ and second‐order statistical texture features of all CT slices of a patient, which means feature extraction was done in 3D. All CT slices of a patient with COVID‐19 must be considered as one data and the infection may appear in some CT image slices and not be shown in other slices. So, one slice alone cannot indicate the stage of COVID‐19 lung involvement of a person, and the sum of features from all CT images must be considered for each person to classify the patients into four classes depending on the percent of pulmonary parenchymal involvement. Also, we have improved the results by finding the best classifier with RF. The advantages of the proposed method with RF, such as great classification rate and high‐speed calculation, make it applicable for medical classification problems. Aggregation result of many decision trees can lead to a more generalized outcome. In other words, when different predictors try to participate in learning independently, the average result can overcome each individual predictor in terms of accuracy. By utilizing several decision trees built by selected subset of features, the robustness of the classifier significantly improves. Therefore, RF classifier has several superiorities compared with other classifiers. Finally, our method compared with deep neural learning has some advantages and disadvantages. Deep learning methods are computationally very expensive and time‐consuming to train with traditional CPUs. These methods depend a lot on training data. This leads to the problem of over‐fitting and generalization in a small database. Moreover, these networks are black boxes, meaning we cannot know how much each independent variable is influencing the dependent variables. However, with big database, deep learning algorithms can automatically extract significant features in hierarchical manner and classify them directly on raw pixels.

TABLE 4

Comparison of classification results of our work with other studies in the classification of severity of COVID‐19 patients from CT images

Study	Methods or features	Results
Huang et al. 26	Affected lung percentage by 2D U‐Net deep learning	N/A
Shan et al. 27	quantification of infection regions by Deep learning	Accuracy = 85.1%
Tang et al. 37	Infection volume of the whole lung and the volume of GGO and RF probabilities	Accuracy = 87.5%
Shen et al. 38	Affected lung percentage by nontrainable CV	N/A
Our work	RF with GLCM + GLRLMS + GLSZM + Global	Accuracy = 90.95%

Our proposed method has some limitations. Other current factors that effect on severity of COVID‐19 patients such as demographic, vital, blood test, laboratory, background disease (like diabetes, etc.), and clinical features are not considered in this study. In addition to lung CT images, these features can also contain useful information to train classifiers and can be added to enhance the accuracy of this automatic system. It is true that we were able to separate the data well with CT scan images alone, but having these features from the patient can also make the result more accurate and reliable. Moreover, the number of patients must be increased. So, in future works, we want to work on a new larger dataset with more information than lung CT images.

CONCLUSION

We have proposed a new automatic practical system based on lung CT images for diagnosis of severity of COVID‐19 to normal, mild, moderate, and severe based on lung involvement percentage and provide insightful findings successfully. The experimental results on 956 patients indicate the combination of first and second statistical texture features along with an RF classifier has the higher results with an accuracy of 90.95%. So, based on the excellent performance, it can be used as an assistant diagnostic tool for physicians to monitor the progression or response of the disease to treatment.