A transfer learning based deep learning model to diagnose covid-19 CT scan images.

To save the life of human beings during the pandemic conditions we need an effective automated method to deal with this situation. In pandemic conditions when the available resources becomes insufficient to handle the patient's load, then we needed some fast and reliable method which analyse the patient medical data with high efficiency and accuracy within time limitations. In this manuscript, an effective and efficient method is proposed for exact diagnosis of the patient whether it is coronavirus disease-2019 (covid-19) positive or negative with the help of deep learning. To find the correct diagnosis with high accuracy we use pre-processed segmented images for the analysis with deep learning. In the first step the X-ray image or computed tomography (CT) of a covid-19 infected person is analysed with various schemes of image segmentation like simple thresholding at 0.3, simple thresholding at 0.6, multiple thresholding (between 26-230) and Otsu's algorithm. On comparative analysis of all these methods, it is found that the Otsu's algorithm is a simple and optimum scheme to improve the segmented outcome of binary image for the diagnosis point of view. Otsu's segmentation scheme gives more precise values in comparison to other methods on the scale of various image quality parameters like accuracy, sensitivity, f-measure, precision, and specificity. For image classification here we use Resnet-50, MobileNet and VGG-16 models of deep learning which gives accuracy 70.24%, 72.95% and 83.18% respectively with non-segmented CT scan images and 75.08%, 80.12% and 99.28% respectively with Otsu's segmented CT scan images. On a comparative study we find that the VGG-16 models with CT scan image segmented with Otsu's segmentation gives very high accuracy of 99.28%. On the basis of the diagnosis of the patient firstly we go for an arterial blood gas (ABG) analysis and then on the behalf of this diagnosis and ABG report, the severity level of the patient can be decided and according to this severity level, proper treatment protocols can be followed immediately to save the patient's life. Compared with the existing works, our deep learning based novel method reduces the complexity, takes much less time and has a greater accuracy for exact diagnosis of coronavirus disease-2019 (covid-19).

Introduction

The infectious coronavirus disease-2019 (covid-19), causes severe respiratory syndrome was first observed in Wuhan city of China in the end of year 2019 and spread in other countries of the world [1, 2]. The mortality rate is very high for critically ill patients with covid-19 [3]. The spreading of covid-19 continues to grow very fast that results over one billion confirmed cases are reported at the end of Jan 2022 [4]. The best strategy to control and cure this pandemic requires effective diagnosis methods which minimize all types of time delay to take appropriate steps to save lives [5]. The concept followed by Govt. of India of triple ‘T’ which is trace, test and treat, became very popular during the second wave of this pandemic to break the chain of carriers of this virus. Screening all suspected cases and providing proper isolation becomes the primary treatment procedures to control this disease [6]. The pathogen testing done at a laboratory is an accurate test possible to confirm the status of patients. It is still a long-time consuming phenomenon which also has the possibility of false negative results [7]. The people with symptoms like respiratory problems or pneumonia are considered as suspected one. Then some laboratory and other non-laboratory specific procedures are performed to confirm the status of infection like arterial blood gas (ABG) analysis, complete blood count (CBC), and pleural effusion [8], all these procedures need some transportation time. On the other hand, some computer-assisted non-laboratory test techniques, which are based on image analysis using chest X-ray or CT-scan to diagnose the infected region of lungs [9].

Nithila et al. in 2019 proposed a region-based active contour model using the various level set function for segmentation of lungs [10]. In the year 2020, Brunese et al. developed a mechanism to identify coronavirus by analysis of X-ray films [11]. In 2020, Pereira et al. also presented a method for identification of covid-19 with the help of chest X-ray and CT-scan [12]. In 2021, Zhang et al. proposed an improved dense generative adversarial network (GAN) for CT-image segmentation [13]. In 2021, Chakraborty et al. proposed a morphology-based segmentation method for efficient diagnosis of covid-19 [14]. In 2021, Imani et al. proposed a shape and texture characteristics based approach to detect covid-19 [15]. In the year 2021, Gaudencio et al. also suggested an entropy-based approach for evaluation of covid-19 [16] and a multi-layer attention technique using U-Net's gives a clear and accurate clinical overview about soft tissue and blood vessels of lungs [17], with some important properties of CT imaging like cost-effectiveness, a wide range of availability and high spatial resolution, it also has some disadvantages like X-ray-based radiation exposure and as compared to magnetic resonance imaging (MRI), it has inferior soft tissue contrast [18].

During the pandemic situations due to the large processing time requirement reverse transcription-polymerase chain reaction (RT-PCR) test is not a very efficient approach to diagnose whether the particular patient is covid-19 positive or negative, it takes hours to complete the analysis. To avoid this time delay we require an efficient tool to diagnose the status of patient either it is covid-19 positive or negative with minimum time limit. The computer based proposed diagnosis method become very helpful to handle this type of situations which require only few seconds to complete the diagnosis with help of various tools of deep learning method. During the second wave of covid-19, it is observed that the various cases were reported as a false negative, which also becomes the biggest cause of the spread of the virus. As per various research studies, the first week of illness is a very important time period to start the treatment of an infected patient. That indicates the necessity to find the severity level of the patient’s infection in the early stage of illness without any delay. But in the present scenario, only a few methods are available that are able to identify the exact diagnosis of a patient's condition quickly. With the help of tools of digital image processing, a new method is proposed which is very helpful for medical experts to find the exact information about the portion of lung involved in the covid-19 infection. So in this proposed manuscript, we present a novel effective tool that diagnoses the exact condition of a patient with the help of some very popular methods of computer based specific learning approach i.e. deep learning [19]. In this novel method first, we segment the CT scan image with the Otsu segmentation method and after this, we can find the condition of the patient with various concepts of deep learning such as Resnet-50, MobileNet and VGG-16. In this proposed manuscript we also present the histogram analysis to find the patient’s exact condition of infection.

The rest of this manuscript is arranged as: Sect. 2 discusses the various research works related to this proposed work. Section 3 has a discussion about the proposed methods to complete the proposed analysis. In Sect. 4, the results with the proposed method are discussed. At the end of this manuscript, the conclusions of this proposed method are summarized in Sect. 5.

Related work

The simple and effective role of image processing techniques attracts many researchers to make the medical data analysis easy, accurate and time-efficient to improve the level of diagnosis in the health sector. The segmentation process of an image is a simple and popular tool, used in processing medical images to improve the patient medical records for the analysis point of view. Numerous methods of image segmentation are very useful to find the anomalies present in the human body, for example in 2007, Lee et al. suggested a method to find the lungs size deformity by applying segmentation mechanism on CT images [20]. In 2012, Saleh et al. developed a technique to identify and analyse diabetic disease by using segmentation [21]. In this sequence, Ganesan et al. in 2013 presented a review report on pectoral muscle segmentation to find breast cancer [22] and in the same year Chen et al. proposed a semi-supervised procedure for liver CT scan image segmentation [23]. In 2014, Kuruvilla et al. suggested a process to find lung cancer with the help of segmentation of CT-scan images [24]. For the correct diagnosis of acute myeloid leukemia by segmentation of cells, Su et al. suggest a mechanism in 2017 [25]. Therefore, segmentation also plays a vital role to analyse the CT-scan images of covid-19 infected patients, as Wang et al. in 2021 designed a procedure based on segmentation analysis of CT-scan images of patients, to find covid-19 infection area [26]. Saeedizadeh et al. in 2021, suggested a segmentation framework to analyse the infected chest regions of patients [27] and in 2021 Rahimzadeh et al. also proposed a fully automated network to diagnose covid-19 with the lungs CT-scan dataset [28].

Various methodologies like segmentation and classification have been proposed on computer vision to deal with the covid-19 pandemic situation [29]. In the present era, image segmentation is an important method in radiology research. Intensity inhomogeneity in terms of the appearance of artifacts and very low differences in the gray level between distinct soft tissues are the main challenges of the segmentation algorithm. With several approaches of segmentation, various medical images are analysed from the covid-19 dataset to measure its performance to segment the medical image perfectly [30]. Along with the various advantages, segmentation methods have so many issues like (i) there is no general method that can be applied to process a huge and continually increased region of interest (ROI) (ii) slight change in method results measure changes in ROI properties (iii) various distinct medical imaging methods availability (iv) changes in signal homogeneity [31]. These problems associated with segmentation algorithms remain challenging [32].

The majority of covid-19 image data sets are mainly focused on the diagnosis. To diagnose covid-19 images in medical practice, artificial intelligence-enabled systems have been designed by Shi et al. [30]. Some concepts associated with the segmentation of infection in CT-scan images are also analysed [33]. In [34], the manuscript based on computer vision proposes various methods to handle the problems and challenges of the covid-19 pandemic situation.

The medical diagnosis and identification of covid-19 is also not an easy task for experienced doctors and medical staff. To make effective analysis from medical reports, it is required to develop a model with less complexity, high processing speed and low computational power automated algorithms that are easily compatible with the system. In this manuscript, a novel mechanism is proposed that is highly effective and efficient to solve the above issues related to automatic identification of lung infection due to covid-19 with the help of segmentation and deep learning. In this proposed approach the image classification is done on segmented image by using the concepts of deep learning, here the initiated technique is evolved and processed with large size covid-CT-dataset [35]. The proposed technique will be very useful in effective analysis of visual medical reports along with this it can be utilized by developing countries that are fighting against this pandemic condition with their limited resources [36].

Proposed method

In the suggested approach as shown in Fig. 1, to identify the status of a patient whether it is covid-19 positive or negative. First, we segment a CT scan image with the optimized method of segmentation to make the medical image investigation effortless and effective. After efficient segmentation, the image classification is performed using the concepts of deep learning with help of a large size dataset to reach an accurate decision. Here a histogram analysis approach is also performed on the segmented medical images to directly find the indication about the patient’s status about covid-19. To successfully achieve the proposed objective we perform the analysis as discussed below:

Fig. 1

Block diagram of the proposed method

Image analysis

To analyse an image of a real object with a computer system it is required to follow the systematic procedure given in Fig. 1 where each step follows the data provided by its previous step as the input data on which some useful algorithms are applied to improve the required information quality. To recognize the targeted object from a given large dataset first it is required to capture the object data set with the help of some sensing-based device like a camera then we go improve the current image format and make it compatible with pre-processing. Figure 2 comprises all the tasks performed to process a medical image.

Fig. 2

Flowchart to process an image on the computer

To find the exact status of the CT-scan dataset, it is required to first segment the medical image with an optimized segmentation method. The main aim of segmentation is basically the separation of the interesting part from the complete portion under consideration, so first, it requires a simple and efficient method of segmentation that provides conversion of binary images from gray-level images. The thresholding methods are used to convert binary images from grayscale images and the efficiency of the process is totally governed by the selection of optimal threshold levels. The simple thresholding method provides a black pixel value if the pixel intensity is less than a certain fixed constant t, or a white pixel value in case of pixel intensity is higher than that threshold t.

Thresholding methods

Multilevel thresholding is a crucial phenomenon in image segmentation, which provides several features in the medical image processing sector like feature extraction, pattern recognition, disease diagnosis etc. From various image segmentation methods, thresholding-based segmentation schemes attracted researchers due to their higher processing rate, lesser memory space requirement and ease of handling. To select the optimal threshold level, there are so many different algorithms present. A detailed survey report on various thresholding methods and their applications are compiled in the systematic form [37]. The bi-level thresholding is a popular method that has two different regions, which are isolated from each other and the thresholding levels for multilevel thresholding is selected with the help of a non-overlapping bi-level thresholding [38]. Multilevel thresholding is also becoming very popular in the medical sector that provides easy, efficient and accurate analysis of the images related to brain tumours [39-41] along with identification of masses in mammograms [42] and the human brain [43], etc. For an image, the selection of a suitable threshold value for multilevel thresholding-based segmentation is the main challenge. It is important for any system to automatically select an accurate threshold level t. Sezgin et al. in 2004 classify the thresholding process into six different groups [44].

Figure 3 comprises the six different thresholding methods for the digital images which all are based on different techniques. In the histogram shape-based technique the threshold value depends on the valleys, peaks and nature of the normalized histogram. Cluster-based techniques use the gray-level samples in which the region of interest (ROI) and the region of not interest (RONI) is segregated from the complete image as two different regions. Next in entropy-based techniques the analysis of a digital image completely depends on the entropy of the foreground and background region of digitized images. Object attribute-based method uses fuzzy model matching, edge coincidence etc. to analyse matching in between various the gray-level of the digitized images. Next in the spatial method, higher-order probability distribution or correlation between image pixels are used, which results in distinct threshold value selections for each pixel in an image.

Fig. 3

Classification of thresholding methods

The clustering-based multi-model image thresholding module is very commonly used to obtain image subdivisions where on behalf of their gray scale values the pixels are categorized into various regions. In comparison between different segmentation methods, the segmentation method based on thresholding is simple and efficient. So, there are different applications of image processing available such as object detection, feature analysis, in the biomedical field and many others are based on this method [45-48]. If the image thresholding phenomenon is based on a single threshold point, it is popularly known as bi-level thresholding. If the thresholding is based on various threshold levels, known as multilevel thresholding, because of the presence of more than one threshold levels in the multi-level phenomenon, intricacy rises and precision reduces during the searching process [49]. In general, image thresholding approaches are based on parametric and non-parametric processes [50, 51]. The parametric approach depends on the probability density function (PDF) for choosing a region with a larger computational cost. The non-parametric approach depends on between-class variance, error rate and entropy value [52-54]. Due to their accuracy and robustness, all these mentioned techniques are normally preferred for searching threshold levels [55].

Medical image segmentation with thresholding methods

Thresholding based segmentation methods are very simple and effective for medical image analysis. Normally it is used to create binary images. The images obtained with thresholding require smaller storage space and have high processing speed. It is of two types basically, simple thresholding and adaptive thresholding. In a simple thresholding operation first, a particular threshold level is decided. Then the pixels, whose levels are higher than the specified threshold level, are denoted with a standard level. In this proposed manuscript to optimize the simple thresholding, first, we arbitrary select the thresholding at 0.3 (Threshold level = 0.3 × 255 = 76.5, approximately 77) and check the output image quality, here to improve the visuals of segmented image we further increase the thresholding at 0.6 (Threshold level = 0.6 × 255 = 153) and compare both the output segmented images. During the analysis of a medical image, we find that if we select more than one threshold level to segment the medical image, the quality of an output segmented image will improve a little bit as compared to simple thresholding methods. To improve the quality of a segmented image further we apply multiple thresholding algorithms. In this segmentation method, a gray level image is segregated into various different regions on the behalf of multiple threshold values. Selection of more than one threshold level and segmenting the image objects based on various distinct features like different brightness levels of objects, for a particular image, are the most challenging part of this method.

To resolve this problem, a segmentation method is required, which has self-updating characteristics to update the threshold level. Otsu's algorithm, named after Nobuyuki Otsu, is the method to convert grey level images to a binary one is a method of clustering-based image thresholding. In Otsu’s, the grey levels which are within-class variance are minimum or maximum, is the desired threshold level [56]. Let [0, 1,…, L– 1] are the L different grey levels of the pixels of a given image. Here ni is defined as the number of pixels at level i and N = n0 + n1 + … + nL–1 is total pixel number. The probability distribution of a normalized image histogram is represented as:

Here the pixels of an image are classify into background and foreground as two classes C0 and C1 which is bifurcated by threshold level t; C0 denotes pixels with levels [0,.…, t], and C1 denotes pixels with levels [t + 1, …, L–1]. Then the class occurrence probabilities and the class mean levels are respectively as:andwhereand

It represents the zeroth and first order cumulative moments of the histogram up to the tth level, respectively, and it represents the total mean level of the image under consideration. Here we can easily justify these relations for any choice of t. The class variances arehere, we use the below discriminate criterion measures for analysis as shown in Eq. (12).

Now this is converted into an optimization problem to decide an optimum threshold t that maximizes the object functions Eq. (12).

The optimal threshold level t* that maximizes , is decided by the sequential search for different threshold levels t from 0 to L–1 with the help of simple cumulative quantities Eqs. (6) and (7), or by using Eqs. (2)– (5): and the optimal threshold t* is

Nobuyuki Otsu designed Otsu’s method [57] which is totally different from the method given by Fisher’s discriminant study [58]. This technique performs better due to the selection of a perfect threshold level for an image that maximizes inter-class variance and minimizing intra-class variance of an image under consideration. Different methods have already been proposed to update the conventional Otsu’s procedure [59-61]. A preferable threshold value finalized based on 2D histogram patterns, Liu et al. introduced a 2D Otsu’s method [62]. Jing et al. [63], suggested the 3D Otsu’s function that achieves the conditions of required variance among pixel clusters for 3D histograms.

In this proposed manuscript, four segmentation algorithms have been applied on the object image and a comparative analysis is performed, on the basis of this we find that the Otsu's method is a simple, self-updating and globally demanded thresholding method, which has a very ample scope [64, 65]. Here Otsu’s segmentation method is basically used to improve the quality of visuals in the reference of analytical aspects which gives an optimum performance for image classification.

Image classification

Image classification, often referred to as image recognition, is basically the phenomenon of assigning a single or multiple labels to a given image. In this proposed approach as shown in Fig. 1 we use concepts of transfer learning on various models of deep learning like Resnet-50, MobileNet and VGG-16 for the classification of segmented CT scan images of patients. To achieve this proposed goal first we introduce some attributes in all above pre-trained models according to the requirement of covid-19 CT scan images. To make this classification exact and highly accurate we trained all these modified models with a large covid-19 CT scan image dataset. After the training and validation process we input the segmented CT scan image on these model and find that the accuracy of CT scan images segmented with Otsu’s segmentation method is higher than the other segmentation methods under consideration to find the status of a patient either it is covid-19 positive or negative.

Results and discussion

A Comparative performance analysis is presented in this section. To confirm the optimal method for analysis, most of the above discussed approaches have been analyzed using various image quality parameters. This analysis is employed on CT-scan images for all approaches using basic quality parameters derived through concepts of confusion matrix [66].

Here, TP refers to true positive means that the actual status and detected status both are correct; FP refers for false positives means the actual status of patient is COVID-19 negative but the detected status shows that the patient is infected with COVID-19; FN refers for false negative means the actual status of patient is COVID-19 positive but the detected status is wrong; TN stands for true negatives means that the actual status of patient is COVID-19 negative and the detected status is correct. To find an optimum segmentation method we apply this method on a large size data set which comprises 650 CT-scan images and out of which twenty CT-scan images are presented as samples to visualize the process exactly.

Medical image segmentation

The segmentation is the basic phenomena to filter out useful information from an image dataset. To find an appropriate method to segment a medical image to extract the useful information regarding the patient's medical diagnosis is not an easy task. From various region-based segmentation methods here we apply two very popular medical image segmentation methods as, clustering and thresholding-based methods [44] to segment an CT scan image. Here on the basis of various image quality parameters like accuracy, precision and sensitivity, we make a comparative study to choose which one is best to segment an CT scan image [66].

From Table 1 we clearly observe that the Otsu which is a thresholding-based segmentation method have precision 95.04% and specificity 69.65% with an accuracy 95.52%. Based on these image quality parameters, we find that.

Table 1

Comparison of various image quality parameters

Segmentation Method	Accuracy	Precision	Specificity
KCM	0.8911	0.8369	0.7664
FCM	0.6037	0.5916	0.0510
OTSU	0.9552	0.9504	0.6965

Implementation

Otsu thresholding method is one of the optimum methods to segment CT scan images of a covid-19 infected patient. To optimize the effectiveness of the Otsu method we also compare this method with some other thresholding schemes in the next step and compare all image performance parameters.

In Figs. 4 and 5, column (a) shows all images in original form, column (b) has the segmented images using simple thresholding at 0.3, column (c) represents simple thresholding at 0.6, column (d) represents multiple thresholding (between 26–230) and column (e) represents Otsu’s-optimal segmented images. To find an optimum segmentation method we compare all the above segmented images on the basis of different image quality parameters.

Fig. 4

Output images using segmentation methods from CT-1 to CT-10

Fig. 5

Output images using segmentation methods from CT-11 to CT-20

From Table 2, it is clear that the Otsu’s method gives the best result for analysing the covid-19 patients’ CT-scan images. A comparative study on various segmentation schemes on the basis of some other performance parameters like MAE, MSE, PSNR, IEF and SSIM [67] with 650 datasets compiled in Table 3.

Table 2

A comparative analysis of various segmentation schemes on an average using 650 CT-image data set

Image Quality Parameter	Simple Thresholding at 0.3	Simple Thresholding at 0.6	Multiple Thresholding	Otsu’s Method
Accuracy	0.95524	0.9978	0.86084	0.9982
Sensitivity	1	1	1	1
F measure	0.9745	0.9987	0.92304	0.99894
Precision	0.95046	0.99744	0.85738	0.99788
MCC	1.32E + 09	1.84E + 09	2.70E + 08	1.85E + 09
Dice	1.19E + 10	1.09E + 10	9.13E + 09	1.09E + 10
Jaccard	-1	-1	-1	-1
Specificity	0.69658	0.98462	0.1598	0.9875

Table 3

Average values of different image quality parameters

Image performance parameter	MAE	MSE	PSNR		IEF	SSIM
Simple thresholding at 0.3	60.98596	75.58642	12.90906		0.0947	0.70784
Simple thresholding at 0.6	60.97922	75.58456	15.56288		0.06102	0.71834
Multiple thresholding	60.84372	75.56786	6.8101		0.32134	0.53736
Otsu’s Method	60.91406	75.57488	15.63732		0.05712	0.72046

From Tables 1, 2 and 3, we can clearly observe that Otsu’s segmentation technique is an optimum method for analysing covid-19 CT-scan datasets. From Table 3, it can be analysed that for multiple thresholding methods, the mean absolute error (MAE) and mean square error (MSE) has minimum values. Along with it, image enhancement factor (IEF) has higher value but peak signal to noise ratio (PSNR) and structure similarity index (SSIM) have extremely lower values. Hence, this method is not preferable. For Otsu’s method, the significant parameters like PSNR and SSIM have maximum values as 15.63732 and 0.72046, respectively, with optimized values of other parameters like MAE, MSE and IEF as 60.91406, 75.57488 and 0.05712, respectively. Hence, with this analysis, it can be confirmed that the Otsu’s method is the perfect method to analyse these CT-scan images.

Figure 6 represents the various image quality parameters of Table 2 as accuracy, F-measure, precision, mcc, dice, specificity, and Fig. 7 represents image performance parameters like MAE, MSE, PSNR, IEF, SSIM of Table 3 in graphical presentation to easily visualize all these parameters.

Fig. 6

Various image quality parameters of an CT image like: (a) Accuracy, (b) F measure, (c) Precision, (d) MCC, (e) Dice, (f) Specificity

Fig. 7

Various segmentation methods on the basis of some image performance parameters like (a) MAE, (b) MSE, (c) PSNR, (d) IEF and (e) SSIM

On comparative analysis of these segmentation methods using various image quality parameters like accuracy, sensitivity, f measure, precision, mcc, dice, Jaccard, specificity etc. and with different image performance parameters as MAE, MSE, PSNR, IEF, SSIM etc., it can be said that Otsu’s is a best suited segmentation method for digital medical images.

Image classification is basically the process to categorize the object image into a particular label. In this proposed manuscript to achieve high classification accuracy the concepts of deep learning are applied [68]. Deep learning is basically a specialized field of machine learning which can be used to train the machine with an unlabelled dataset, here the training process simply governs with learning with a given dataset. The deep learning approach of image classification totally depends on the training dataset used for training purposes. Sometimes the performance of the computer becomes more efficient than human beings to classify an image dataset due to the high accuracy achieved during the training process. To find an appropriate result at output we apply the concepts of deep learning on the images segmented with Otsu’s segmentation as shown in Fig. 8.

Fig. 8

Deep learning approach for medical image classification

In this proposed manuscript we consider three very popular models of deep learning as Resnet-50 [69], MobileNet [70] and VGG-16 [71] for classification of a covid-19 CT scan images, which gives exact status of a patient that, either it is covid-19 positive or negative with very high accuracy.

From Table 4 and Table 5 we can observe clearly that Resnet-50, MobileNet and VGG-16 give accurate diagnosis with all segmented CT scan images. For non-segmented CT scan images, the Resnet-50 model gives wrong predictions for CT-1, CT-4 and CT-11 and the MobileNet model gives wrong predictions for CT-1 and CT-11 only. For CT scan images segmented with Otsu’s method these three models give correct prediction for all CT scan images from CT-1 to CT-20, hence we can say that all these three models perform well with the Otsu’s segmented CT scan images. Here the VGG-16 model gives all correct predictions with segmented and non-segmented CT scan images both with very high accuracy as shown in Table 6.

Table 4

The status of covid-19 for CT scan images from CT-1 to CT-10. (*P- Positive, *N-Negative)

Table 5

The status of covid-19 for CT scan images from CT-11 to CT-20. (*P- Positive, *N-Negative)

Table 6

A comparative analysis of image classification with deep learning models on the basis of accuracy

CT No	Covid-19 Status	Accuracy without Segmentation			Accuracy with Otsu Segmentation
		Resnet-50	MobileNet	VGG-16	Resnet-50	MobileNet	VGG-16
CT-1	Positive	62.99	65.45	68.79	71.12	74.21	98.89
CT-2	Positive	70.17	73.32	84.24	73.68	76.58	99.35
CT-3	Positive	72.22	75.69	94.50	75.64	78.13	98.50
CT-4	Positive	69.36	69.93	73.02	73.14	75.19	97.51
CT-5	Positive	73.46	76.32	96.69	76.21	78.51	99.86
CT-6	Negative	70.12	72.03	79.96	76.87	76.54	99.96
CT-7	Negative	71.23	76.14	89.59	77.13	81.02	99.99
CT-8	Negative	73.83	78.25	94.13	76.15	80.24	99.93
CT-9	Negative	71.41	74.63	88.49	74.56	78.36	99.99
CT-10	Positive	70.23	73.41	89.92	73.56	79.89	99.96
CT-11	Positive	61.32	63.99	72.94	71.59	69.15	98.96
CT-12	Negative	70.89	73.63	87.62	76.51	88.29	99.69
CT-13	Negative	69.81	71.28	77.80	70.98	81.29	99.80
CT-14	Positive	70.23	75.63	88.67	73.53	80.29	99.62
CT-15	Positive	71.81	75.65	86.97	74.52	81.28	98.37
CT-16	Positive	72.31	74.25	87.92	78.62	89.14	99.90
CT-17	Negative	71.89	73.21	82.14	79.31	81.14	98.70
CT-18	Positive	70.16	73.32	76.15	78.52	87.23	98.17
CT-19	Positive	70.07	70.99	72.19	72.69	87.53	99.77
CT-20	Positive	71.32	71.98	72.14	77.32	78.52	98.74

Medical images analysis with histogram

The histogram representation is very helpful to analyze the medical images like CT-scans or X-rays which contains all the information regarding the medical issues in black and white region format [72]. Histogram of a medical image represents the repetition of a particular pixel value in the gray scale images. The histograms are very helpful for segmentation of binary or gray level medical images to find clear visualization for analysis. The histogram analysis of original image and segmented image with Otsu’s segmentation method is shown as below:

Figures 9 and 10 comprises the histogram of a small sample set of twenty original grayscale CT-scan images and corresponding Otsu’s segmented images. The analysis of corresponding histogram representation from CT-1 to CT-20, it is clear that a patients those are found covid-19 positive (CT-1, CT-2, CT-3, CT-4, CT-5, CT-10, CT-11, CT-14, CT-15, CT-16, CT-18, CT-19, CT-20) their histogram shows an histogram equalization condition at central portion and have a sharp rise at both corners.

Fig. 9

The histogram representation of original and Otsu segmented CT scan images from CT-1 to CT-10

Fig. 10

The histogram representation of original and Otsu segmented CT scan images from CT-11 to CT-20

For covid-19 negative patients (CT-6, CT-7, CT-8, CT-9, CT-12, CT-13, CT-17) the histogram shows that only pixels of smaller and higher values are present at the corner sides and the pixels of middle size valves are totally absent. Hence we can say that with the help of histogram analysis of Otsu’s segmented images we can also judge the status of a patient whether it is covid-19 positive or negative.

From Table 6 it is clear that the deep learning models for image classification perform efficiently to give the correct status of covid-19 patients. In this proposed method we use Resnet-50, MobileNet and VGG-16 models for image classification which all give an optimum accuracy with all CT scan images. Here we find that all these three models perform well with Otsu segmented images as compared to original CT scan image. In the above analysis we also find that the VGG-16 model is performing very well as compared to Resnet-50 and MobileNet models of deep learning.

From Table 7 it found that the overall accuracy of Resnet-50, MobileNet and VGG-16 on non-segmented CT scan images is 70.27%, 72.95% and 83.18% respectively which ensure that all three models can gives the correct result about the status of the patient that either is positive or negative. With the Otsu-segmented CT scan images these three models Resnet-50, MobileNet and VGG-16 gives 75.08%, 80.12% and 99.28% accuracy respectively, which shows that the VGG-16 model is best suited for the analysis of Otsu’s segmented CT scan images to find out the status of a patient with the help of computer vision.

Table 7

Analysis of image classification with Resnet-50, MobileNet and VGG-16 on the basis of accuracy

Deep Learning Model	Accuracy without Segmentation	Accuracy with Otsu Segmentation
Resnet-50	70.24	75.08
MobileNet	72.95	80.12
VGG-16	83.18	99.28

Figure 11 represents the graphical representation of the deep learning approach for image classification, from this graphical representation it is clear that all three models Resnet-50, MobileNet, VGG-16 are efficient to find the correct diagnosis of a patient. When we apply all these models on segmented images the accuracy is improved to some extent. Here it is very clear that the VGG-16 model with Otsu segmented CT scan images gives highest accuracy i.e. 99.28% so it is best suited model to detect covid-19 cases.

Fig. 11

The graphical representation of the accuracy for Resnet-50, MobileNet and VGG-16 models

Figure 12 represents the complete analysis in pictorial form using optimized tools of image processing and deep learning on the basis of various quality and performance parameters. This proposed model makes the diagnosis of covid-19 suspected patient more simple, accurate, efficient and reliable. This complete diagnosis can also be evaluated with the help of histogram analysis.

Fig. 12

The pictorial representation of result for the proposed model

In Table 8 some recent segmentation based approaches are compiled to diagnose covid-19. In 2020, Zheng et al. suggested a method which is based on 499 CT volumes with the U-Net approach. In the same year (2020) Cao et al. also proposed an approach with U-Net architecture which is based on 2 patient dataset. In 2020 Gozes et al. also develop an approach with deep learning CT image based on a testing set of 157 patients. In 2020, Jin et al. analyzed 723 positive cases using UNet along with CNN. Using ResNet-50 in 2020 Ying et al. diagnose covid-19 status of 88 patients. In 2020, Shen et al. proposed a threshold-based region growing approach with 44 confirmed covid-19 cases. In comparison with all above research approaches it can be clearly found that the proposed method which is based on pixel level analysis of segmented medical image with Otsu’s method, addressed various challenges of related research work by making proposed method simple, robust, user-friendly and time efficient which provides exact and highly accurate status of a patient either it is positive or negative within a minimum time frame.

Table 8

The summary of few segmentation-based approaches to diagnose COVID-19

Authors	Year	Database used	Approach used
Zheng et al. [73]	2020	499 + 131 CT volumes	U-Net
Cao et al. [74]	2020	2 Patients	U-Net architecture
Gozes et al. [75]	2020	Testing set of 157 patients	Deep learning CT image analysis
Jin et al. [76]	2020	1136 cases 723 positives	UNet +, CNN
Ying et al. [30]	2020	88 patients diagnosed with the COVID-19	ResNet-50
Shen et al. [77]	2020	44 confirmed COVID-19 cases	Threshold-based region growing
Proposed work	2022	650 patient’s dataset out of which 350 is covid-19 positive	Otsu’s based deep learning approach with Resnet-50, MobileNet and VGG-16 models

Conclusion

In this paper, a deep learning approach is proposed to diagnose the status of a covid-19 infected patient within a minimum time frame with high accuracy. This method is very helpful to medical experts to find out the actual condition of a patient and on the behalf of this, they can follow the required treatment protocol without any time delay. In this context, segmented image is obtained by employing the Otsu’s segmentation method. To select an optimized segmentation method, various segmentation methods have been analysed with different quality parameters like accuracy, sensitivity, f measure, precision, mcc, dice, Jaccard and specificity and it is determined that Otsu’s is a suitable method to segment a medical image. To confirm the suitability of Otsu’s method to analysed a digital medical image, here we analysed the Otsu’s method with various performance measures also like MAE, MSE, PSNR, IEF, and SSIM then find that the Otsu’s method is best suited in reference to medical image segmentation. For the classification it is clearly found that VGG-16 gives the high accuracy 99.28% with segmented CT scan image dataset and 83.18% accuracy with non-segmented CT scan image dataset. Hence, we can say that the performance of VGG-16 is much better in comparison with Resnet-50 and MobileNet for CT scan image dataset. On the basis of the exact diagnosis of the patient, if the severity level of a patient is critical then doctors may proceed for ABG and CBC analysis to find the actual current medical condition of that patient and provide them all treatment protocols required, as per the diagnosed medical condition. The proposed method is also helpful to minimize the crowd of patients in the holding area, covid-19 wards, and in intensive Care Units (ICUs) of the hospitals. It also saves lots of precious time for health workers. As a part of future work, the above diagnosis procedure can be further extended to attain more accuracy with some other models of deep learning which can also become useful to diagnose some other disease related to the human body.