Res-CovNet: an internet of medical health things driven COVID-19 framework using transfer learning.

Major countries are globally facing difficult situations due to this pandemic disease, COVID-19. There are high chances of getting false positives and false negatives identifying the COVID-19 symptoms through existing medical practices such as PCR (polymerase chain reaction) and RT-PCR (reverse transcription-polymerase chain reaction). It might lead to a community spread of the disease. The alternative of these tests can be CT (Computer Tomography) imaging or X-rays of the lungs to identify the patient with COVID-19 symptoms more accurately. Furthermore, by using feasible and usable technology to automate the identification of COVID-19, the facilities can be improved. This notion became the basic framework, Res-CovNet, of the implemented methodology, a hybrid methodology to bring different platforms into a single platform. This basic framework is incorporated into IoMT based framework, a web-based service to identify and classify various forms of pneumonia or COVID-19 utilizing chest X-ray images. For the front end, the.NET framework along with C# language was utilized, MongoDB was utilized for the storage aspect, Res-CovNet was utilized for the processing aspect. Deep learning combined with the notion forms a comprehensive implementation of the framework, Res-CovNet, to classify the COVID-19 affected patients from pneumonia-affected patients as both lung imaging looks similar to the naked eye. The implemented framework, Res-CovNet, developed with the technique, transfer learning in which ResNet-50 used as a pre-trained model and then extended with classification layers. The work implemented using the data of X-ray images collected from the various trustable sources that include cases such as normal, bacterial pneumonia, viral pneumonia, and COVID-19, with the overall size of the data is about 5856. The accuracy of the model implemented is about 98.4% in identifying COVID-19 against the normal cases. The accuracy of the model is about 96.2% in the case of identifying COVID-19 against all other cases, as mentioned.

Introduction

With the global pandemic crisis, a huge number of the Internet of Medical Things (IoMT) devices and sensors are being used for the detection and surveillance of COVID-19 [1-3]. COVID-19 is triggered by the extreme severe respiratory syndrome coronavirus two (SARS-CoV-2). The mortality rate is cultivating worryingly across the globe, needing an earlier detection response and curtailing this particular ailment’s fast spread. A crucial stage with the battle counter to COVID-19 is identifying the infected individuals through screening efficiently. The affected individuals can obtain immediate care and medication which acts as a key to diminishing the pandemic spread [4]. Because of the lack of good therapies and medications, the state is now appalling for vast amounts of people. In COVID-19 patients, symptoms range from fever, sore throats, dry cough to organ failure, extreme pneumonia, septic shock, and identified Acute Respiratory Distress Syndrome (ARDS), affecting the lungs of the individuals [2]. The numbers related to COVID-19 cases also reveal that the effect of COVID-19 globally, and these statistics considered from Johns Hopkins University’s developed dashboard up to December 4, 2020, as mentioned in Fig. 1 [5].

Fig. 1

Representation of trends in COVID-19 cases a COVID-19 cumulative cases b COVID-19 cumulative deaths [5]

The primary screening technique employed for COVID-19 identification in infected individuals is polymerase chain reaction (PCR) [6] testing. This test is one of the options for detecting SARS-Cov-2 RNA traces in accumulated breathing specimens collected through a variety of methods such as oropharyngeal or nasopharyngeal swabs. The test, PCR, is highly vulnerable, so its assessment can be considered the gold standard due to its time, laborious process, and critical hand system, which is not that abundant in supply. Another diagnostic technique to detect COVID-19 is Reverse Transcription polymerase chain reaction (RT-PCR), which is a more commonly used diagnostic technique. Still, it has low sensitivity problems in initial phases with lengthy assessment facilitating additional transmission [7]. Thus, most people with suspected pneumonia symptoms are recommended to scan the chest by utilizing the techniques, X-rays, and Computer Tomography (CT) scans for faster diagnosis and isolation of the affected people [8]. The chest region radiography imaging, which involves diagnostic techniques such as X-ray or computed tomography (CT) imaging, can be performed and analyzed by radiology specialists to look for variations that suggest the viral infection linked to SARS-CoV-2 visually, is an alternative assessment technique that has been used for COVID-19 screening. Due to the limited expertise available, the necessity of getting great comparisons of COVID-19 to regular pneumonia, an automated detection program dependent on artificial intelligence (AI) might be a fundamental event toward a significant decrease in examining duration. In [9-11], CT scans are utilized for automated COVID-19 pneumonia detection utilizing heavy learning-based systems.

Even if better quality data can be obtained using the CT scan technique, the technique using X-rays is more quickly, a lot easier to manage, much less harmful, and more economical options. However, the COVID-19 affected individuals’ X-ray reports are in shortage due to that the effective instruction of a rather rich network is tough [12]. In this particular situation, transfer-learning may also be considered a practical answer. Nevertheless, the standard transfer learning program that utilizes the ImageNet data repository can be pre-trained with the aid of proven deep networks for a successful pass. It may not be a good choice at the very initial stages as the qualities of COVID-19 X-ray reports vary exclusively with the other reports as they were designed with the perception of utilizing for other scenarios. To better understand the situation, the usage of the chest X-rays for the daunting undertaking of distinguishing individuals with normal viral or bacterial infection from COVID-19 individuals with huge overlapping features has not yet been posted [13]. This study introduces Res-CovNet, a powerful convolution neural community layout with the assistance of X-ray reports, guided by the urgent need for a solution to help in the fight against the severe pandemic, COVID-19, and stimulated by the accessible open-source as well as open-access attempts by the scientific research world.

The framework demonstrates the maximum advantage utilizing the dataset to impart Res-CovNet is composed of 6006 posteroanterior chest radiography X-ray images across 4,423 affected person instances that include viral pneumonia, bacterial pneumonia, and COVID-19 cases. Moreover, the designed framework scrutinizes how far Res-CovNet can make the predictions with the technique of explainability to get greater intuitions into crucial aspects of COVID-19 that might help clinicians enhance the screening [14]. This paper proposes an effective method based on X-ray images with the data availability to acclaim an effective framework with the help of deep neural networks such as the professional information that could be used effectively to detect COVID-19 cases despite the relatively limited sizes of available COVID-19 X-rays. In the initial stage, rather than utilizing different standard sources that are utilized for different functions, a wider database with X-rays from frequent along with other non-COVID pneumonia patients is utilized for mastering transfer [15]. A genuine and significant neural network named Res-CovNet is suggested to discover COVID-19 from X-rays that developed based on the transfer learning technique by utilizing the popular conventional convolution neural network architecture such as ResNet-50 for the pre-training purpose.

Additionally, a stack of classification layers is added after the ResNet-50 for the prediction enhancement of various X-ray resolutions to protect several open disciplines. With the aid of extra tunning of layers finely, the sophisticated convolution layers are then transmitted precisely to learn together with different X-ray reports on the smaller COVID-19 X-rays data. This updated system combines all of the original X-ray learning into a lot more COVID-19 based X-rays research for the identification appropriately—robust tests of the suggested framework to demonstrate good results in the most standard evaluation metrics.

The focused contributions of the proposed research can be mentioned as follows:

Utilizing the IoMHT (Internet Medical Health Things) for the collection of bio-medical images that are necessary to identify COVID-19.

The information collection was made through the recently popular database MongoDB.

With the aid of biomedical images and deep learning methodologies, identifying COVID-19 with higher reliability.

With the aid of biomedical images and deep learning methodologies, differentiating various types of Pneumonia and COVID-19 with higher reliability.

The latest transfer learning concept is utilized in determining the type of pneumonia, or COVID-19.

The manuscript is organized as follows: Sect. 2 covers the related work, the Sect. 3 covers the methodology that includes the concept of convolutional neural networks and their popular architectures, transfer learning and their advantages and challenges, proposed Res-CovNet architecture, and Proposed algorithm, the Sect. 4 covers the data description, an analysis of results that include system requirements, and performance assessment, lastly, in Sect. 5, the conclusions are made, and the future works are also mentioned.

Related work

A large amount of data (e.g., IoT devices, sensors, pandemic data) related to COVID-19 patients are produced by the extensive use of IoMT applications, it is essential to analyze, and automatically process this data without much intervention of human. In this regard, AI along with IoT has the potential to do such tasks [16-24]. An EEG-based pathology detection system using different layers of CNN fusion was proposed in [25]. Different types of fusion networks such as MLP (multi-layer perceptron) and autoencoder were investigated. The system achieved high accuracy and the bandwidth consumption was less. Sohrabi et al. [1] proposed a Capsule Networks based framework to identify COVID-19 illness by using X-ray images. In this work, many convolution levels and capsules are composed to overcome the problem of class imbalance. In experimental analysis, it proved the gratifying overall performance with a limited amount of trainable parameters. The dataset considered trained design is publicly available on Github for open access. It was regarded as the first three cases of COVID-19 influenced situations in France. Out of these three people, 2 ended up being determined in Paris and one in Bordeaux. Before coming in contact with COVID-19 diseases they'd been staying in Wuhan, China. Chih-Lai et al. [2] explained the post scenario of COVID-19 outbreak conditions mostly in the case of china. Also, discussed the clinical tests of the vaccines, the type of datasets available such as X-rays, CT scan reposts, possible treatment alternatives, COVID-19 related statistics, prevention and control of the disease, and unsettled issues.

Rothan and Byrareddy [4] presented a hybrid plan dependent on AI, which uniquely made use of ML and DL algorithms. The suggested strategy is particularly used for detecting COVID-19 cases utilizing chest X-ray image data. The framework has supplied a radiologic analysis of MERS (Middle East Repository Syndrome) on novel coronavirus. They believed the situation of 30 years old age male patient who endured diarrhea, fever, and abdominal pain. The authors offered an analysis of the treatment of infected people with chest X-rays. Further, they utilized the device on a collected dataset of chest X-ray and CT photos as well as experienced enhanced results. Mohammed et al. [7] proposed a methodology for automatic recognition of COVID-19 infected person from the healthy person using X-ray images. This methodology was built using two reliable technologies they include conventional machine learning methodologies and deep learning frameworks [26, 27]. The deep learning framework utilized for this proposed research is ResNet50 and the proposed model able to attain an accuracy of 98.8%. Xie et al. [28] explained the aspect of the impact of radiography imaging concerning the traditional clinical method RT-PCR. The document also claimed that the method RT-PCR provides false negatives will be possible but with the case of radiography imaging, it can be resolved as better accuracy in identifying the disease.

Gozes et al. [9], described that the sort of protocols must follow by hospital staff members to decrease the risk of nourishing people as well as what security measure is needed in preserving COVID-19 infected patients. Li et al. [10], noted the outbreak of etiology in China group Wuhan. Furthermore, they nurtured a query about the actual reason behind this epidemic. In this investigation, they look at the traveling (via a company as well as air) impact on COVID-19. Mohammed et al. [29] identified the necessity of benchmarking for the various methodologies that are still evolving for the automatic recognition of COVID-19 infected person from a healthy person. The proposed study was framed in the form of a decision matrix that includes 10 assessment criteria and 12 automatic COVID-19 recognition methodologies and this methodology was termed as Multi-Criteria Decision Making (MCDM). TOPSIS is utilized to benchmark as well as the generation of ranking among the various methodologies. This study showed that the linear SVM-based methodology is the best one with the coefficient of closeness is 0.9899.

Vaid et al. [30] developed a methodology by utilizing deep learning methodologies by using chest X-ray reports. This framework was mainly developed based on the convolutional neural networks and transfer learning methodology to identify the malformations structurally and classification of diseases by using a dataset, besides, small in size. The developed methodology provided about 96.3% accuracy. Besides, the results looking better but the dataset is very small. For this type of framework, validation is required. Al-Waisy et al. [31] proposed a methodology for the automatic recognition of COVID-19 with the aid of X-ray images. The proposed framework was termed as COVID-CheXNet. The proposed methodology was implemented in two steps. The first one deals with various image processing techniques to obtain a clear x-ray image. The second step deals with the identification of COVID-19 with the pre-trained deep learning framework along with ResNet34. The proposed framework is able to attain an accuracy of 99.99%, precision of 100%, recall rate of 99.98%, and F1 score of 99.99%.

Apostolopoulos and Mpesiana [6] assessed the CNN-based model along with the transition learning performance in detection COVID-19 affected individuals. The dataset considered for this study consists of X-ray reports with various category individuals such as normal, pneumonia, and COVID-19 cases collected from various medical public sources. The significance of the framework is explained with the help of various statistics such as sensitivity with 98.66%, specificity with 96.46%, and the accuracy obtained about 96.78%. This framework generates hope for the future to detect COVID-19 effectively without having any human contact along with maintaining social distance. Muhammad and Hossain [32] introduced a deep learning CNN approach for predicting and detecting COVID-19 patients and Non-COVID-19 patients based on their Lung ultrasound (LUS) images. They got an average accuracy of 91.8%. Experiments are carried out with LUS images and video datasets that are publicly available. Shorfuzzaman and Hossain [33] chest described deep meta-learning-based detection of COVID-19 using X-ray (CXR) images. Tiwari et al. proposed TermInformer to mine the disease and their related cure which applied for COVID-19 detection too [34].

Wang et al. [35] implemented a deep learning full automated system on the dataset collected from 7 cities. Datasets considered were CT-EGFR which contains 4106 patient’s lung cancer gathered from West China Hospital of Sichuan University and another dataset is COVID-19 which is collected from six hospitals in China. On the training set, they got an accuracy of 81.24%. Shi et al. [36] proposed a method for weakly supervised learning for COVID detection using Convolutional layers. This method minimizes the manual labeling of CT-images and can get the same results with fewer details. Dataset gathered from 2 hospitals a total of 420 CT images and got an accuracy of 81%. Jamshidi et al. [37] described various AI-based techniques and deep learning approaches for the COVID-19 issue which includes GAN, RNN, etc. The document provided an overview of the different image techniques, geographical temperature, and locality issues which further elaborates on various ways to solve in diagnosis and treatment of COVID-19.

So far various versions of discussions on COVID-19 were seen. Some researchers explained the real-time situation with COVID-19 was discussed. Some researchers argued that traditional technique like RT-PCR provides false negatives with high possibilities, so the possible alternative suggested as radiographic imaging with real-time information. The remaining researchers explained the various methods particularly with the help of deep learning and machine learning techniques on radiographic imaging (X-rays and CT scan) to move into an AI-based platform to identify COVID-19 disease without having human interaction.

Methodological discussion

This section focuses on the different methodologies that are used in the proposed system, either directly or indirectly. The generalized structure as well as the functioning of CNN and mathematical expressions that describe the various phases, some common CNN models that deal with disease diagnosis using various types of images, the principle of transfer learning, and finally the proposed methodology are discussed in this section.

Generalized architecture and functioning of convolutional neural network

One can consider the Convolutional Neural Network, popularly known as CNN, as a great combination of genetics, architecture, and mathematics due to certain reasons such as their resemblances with neural networks as that of the brain, the complex structure of the networks, the operation behind this network with powerful mathematical support. There is a competition through a hackathon called ImageNet 2012 that brought Convolutional Neural Networks into the public scenario. After then, the rigorous work on deep learning models along with CNN observed very commonly due to its accuracy provision when compared to other models of ANN or machine learning. Thus, that resulted in expanding its wings into various fields such as Face recognition, object identification, recommendation systems and analysis, and review of documents.

Architecturally, the CNN model can be made into various segments such as the convolutional layers, the pooling layers, and the fully connected layers. The convolution layers mainly deal with the feature extraction from the provided input image, the pooling layers mainly deal with the reduction of various aspects such as spatial representation, number of parameters, and the computations aspects related to the network, and the fully connected layers mainly deal with the generation of output from the obtained information of the convolution layers. The whole process of CNN can shortly be explained in the following Fig. 2.

Fig. 2

Sample architecture of convolutional neural network (CNN)

For all the deep learning algorithms, the heart of those methodologies is the Neural Networks. Three actions take place through the neural network such as input obtaining, data processing, and output generation. These three actions are represented in the form of layers in a neural network such as the input layer, hidden layer, and output layer. The training process that takes place in a neural network can be executed in two steps such as the forward propagation and backward propagation. In the case of the forward propagation of CNN, a series of actions take place such as the input data obtained, the data processing, the output generation. While the backward propagation of CNN, a series of actions takes place such as evaluation of error and update the parameters.

Mathematical representation of the forward and Backward propagations in CNN

The major operations in the forward propagation will be done at various layers such as convolution layers and pooling layers. The operation that takes place in the convolution layer can be represented mathematically as mentioned in Eq. 1where X is the input image at the initial stage of the network and in the later stages the output obtained from the previous layers, and f represents filter which is also referred to as kernel. If the dimensions of the image are n-by-n and the dimensions of the filter are f-by-f then the size of the image after convolution will be if f > 1. But, this intimates that the image is getting shrunk which might lead to loss of the data. To avoid such a situation, the padding of zeros to the image at the top, bottom, right, and left can be made. Now, the new scenario with the image size n-by-n, filter size f-by-f, and p indicates the no. of padding rows or columns added on each side of the image than the size of the convolved image can be given as -by-. The values of f, usually considered to be odd, as the center of the filter should exist to modify and the values of p can be generated as mentioned in Eq. 2.

The other type of convolution can also be considered in the case of CNN called the strided convolution. In this scenario, one must mention the number of pixels that will be jumped when convolution is made with a filter or kernel. Suppose, s indicates the stride of the convolution then the size of the image obtained through stride convolution can be given as . This operation is also known as cross-correlation. If we combine stride convolution with padding then the value of p can be given as mentioned in Eq. 3. In Eq. 3, When s = 1, generates the conventional convolution with padding scenario araises.

Besides convolution layers, the CNN architecture utilizes pooling layers for decreasing the size of the input images as well as the number of parameters that further lead to speeding up the computations, and that could show some effect in identifying the features more robust. Pooling can be done in various forms but conventional methods such as the maximum pooling layer and the average pooling layer. The max-pooling mainly deals with retaining the max value of the filter in the filter region to identify the features. The average-pooling is also similar to max pooling but an average of the filter values will be considered instead of max values. The pooling layers are essential in the case of robust architectures like CNN for the reduction of computations and its efficiency in real-time also works well.

The image obtained through the convolution and pooling layers passed on to fully connected layers the dimensions of the image will be reduced then this output will be transformed by using a linear and nonlinear transformation. At first, the linear transformation will be applied in the following way as mentioned in Eq. 4where W is the weight which is randomly initialized, X is the input image, b is the bias. The dimensions of the matrix W are m-by-n, where m indicates the number of features of the matrix X, the dimensions of the matrix X are m-by-1, where m indicates the number of features of the matrix, and the dimensions of the matrix b are n-by-1. Now, the process of nonlinear transformation will be considered as it can capture the more complex features. The function used for this transformation is called an activation function. There are a lot of activation functions that exist some of them can be linear and others will be nonlinear. The examples of nonlinear activation functions are sigmoid, Tanh, ReLu, Leaky ReLu, Parametric ReLu, Softmax, and Swish. These activation functions will be selected depending on the type of case study dealing with.

The backward propagation aids in obtaining more accurate values by updating the values of the parameters which are part of CNN architecture. The propagation direction in this case from output to input. Usually, gradient descent methodology is utilized to update the values of the parameters. The parameters that exist in fully connected layers are weight (W) and bias (b) parameters. To update these parameters, the error needs to be evaluated that can be represented as mentioned in Eq. 5where E represents the error, W represents the weight matrix, O represents the final output in the forward Propagation. The weight (W) parameter can be updated depending on the evaluated by using the mentioned Eq. 6where represents the updated weight matrix, represents the weight matrix before the update, and represents the learning rate. Similarly, the same scenario arises with the parameter bias value update, for proceeding ahead, first need to evaluate by using the mentioned Eq. 7where E represents the error, b represents the bias, O represents the final output in the Forward Propagation. The bias (b) parameter can be updated depending on the evaluated by using the mentioned Eq. 8where represents the updated bias, represents the bias before the update, and represents the learning rate. The parameter in the convolutional layers is the filter that needs to be updated. For updating the filter parameter, first, need to evaluate that can be evaluated as mentioned in Eq. 9.

Depending on the above Eq. 9, the filter parameter can be updated by using the following mentioned Eq. 10.where represents the updated filter, represents the filter before the update, and represents the learning rate.

Some popular CNN models

Some of the popular and standard convolution neural networks depending on their architectures are LeNet-5 [38], AlexNet [39], VGG-16 [40], ResNet [41], Inception [42], LeNet-5 [38] network considered being one of the classic networks, published in 1998. This network was developed for achieving the main aim such as detection of the handwritten digits with a gray image of size 32 × 32 × 1. The number of parameters used by this model is about 60 k. This model phenomenal in decreasing the size of the image as the channel number increases. The pooling layers section of the network is made with the help of average pooling. The activation functions used when this model was published in 1998 were sigmoid and Tanh whereas the latest implementation utilizes ReLu in many of the scenarios.

AlexNet [39] network is also considered being one of the classic networks published in 2012. This network is named after the first author of this work, namely, Alex Krizhevsky. This network was developed for achieving the main aim such as the classification of images into classes of 1000 that was the part of ImageNet Challenge. It is comparably the same as LeNet-5 but it is huge in structure. The number of parameters used by this model is about 60 million. The activation function used in this net was ReLu. The important aspect of this publication that the authors mentioned the usage of multiple GPUs and Local Response Normalization and later it was proved that Local Response Normalization is not that helpful.

VGG-16 [40] network also considered being one of the classical networks published in 2015. This network was published for achieving the modification of AlexNet. It might be based on AlexNet but it is very huge even if someone considers the contemporary standards. The number of parameters used by this model is about 138 million. The activation function used in this net was ReLu and eliminated the concept of Local Response Normalization from this architecture as it was not improving the performance of the network. The number of filters used in this net increasing from 64 to 128 and then to 256 and then to 512. The number of filters 512 used twice in this architecture. The shrinking of dimensions takes place in this network due to the Pooling. There exists another version of VGG-16, i.e., VGG-19 which is bigger than VGG-16 but most researchers hesitate to use VGG-19 as it will be doing the same work as that of VGG-16. The work of VGG-16 was recognized more effectively due to the rules made to use CNN nets.

ResNet [41] is the abbreviated form of Residual Net. The building blocks of ResNet are Residual Blocks which would help produce either shortcut connection or skip connection before the following activation takes place. The basic intention of developing this network was to improve the efficiency in terms of performance by stacking up these blocks of ResNet as training the data using deep neural networks is very difficult due to their diminishing and blowing up the gradient problems. There are two types of residual blocks in ResNet such as Identity Block and Convolution Block. In the case of Identity Block, a block in which convolution block followed by normalization block and that followed by the activation function, ReLu. In this case, the skip connection can be formed for the layers over 2. The case of Convolution Block mainly deals with the network in-network [42] kind of structure. The network in-network structure is utilized in popular CNN architectures such as ResNet and Inception. This structure plays a significant role in diminishing the number of channels that are popularly known as feature transformation.

Theoretically, as one goes deeper into the neural networks that will generate a better solution, but practically, due to diminishing and blowing up the gradient problems, the performance will get affected as one goes deeper into the neural networks. That notion has been changed once ResNet developed as this architecture provides better performance as one goes deep into the ResNet as compared with theoretical knowledge. The activation function used in this net is ReLu.

Inception [42] is also a CNN architecture popularly known as GoogleNet as it was developed by Google. The name Inception was considered from the meme image that is taken from the movie Inception. In this architecture, the pooling will be done by both average pooling and max pooling. The activation function used is softmax. The inception module is responsible for the reduction of the input dimensions that can be used after pooling i.e., max-pooling sometimes in the architecture. To drive in the direction of achieving the aim, there exist three softmax branches at various locations in the inception architecture and also due to these branches’ regularization effect a raise to maintain intermediate features good enough to learn by the network. G.

Transfer learning

Transfer learning mainly deals with training a model on one problem and then utilizes it on the related models as an initial stage. The usage of transfer learning can be explained in a precise way from the perspective of supervised learning i.e., if the problems are of similar nature with the same inputs whereas the targets are different. The benefit of using this methodology is to reduce the time of training for the model and to attain a low generalization error. The deep learning API of python, Keras, helpful in providing CNN models such as VGG-16, VGG-19, ResNet, and Inception for transfer learning. As per the efficiencies, of the popular architectures, ResNet is considered to be one of the bests due to its structural efficiency of the framework when compared to all other models. That is the very reason, ResNet is considered for the proposed model. The general structure of transfer learning represented as mentioned in Fig. 3.

Fig. 3

General structure of transfer learning

Advantages of transfer learning

There are other benefits above and beyond potential benefits in time and resources in utilizing transfer learning. A primary benefit is that an appropriate marking training set is available for the existing scenario of the problem. sAn current model (from a similar problem scenario) can be utilized with extra training to help the modern scenario of the problem where inadequate training data are available. Feature extraction, as well as classification, can be implemented with the aid of deep learning with a smaller topological network. Based on the scenario of the problem, the classification or the output get vary between two different scenarios. Therefore, for every new scenario of classification problem, it is sufficient to make changes in the classification layer which is followed by the pre-trained model. It needs far less time for training and validating the model by utilizing the pre-trained model.

Challenges of transfer learning

The potential of reducing the work needed to develop robust neural networks related to deep learning concepts in connection with transfer learning is not at all new. This has an issue of decrement in the accuracy of the model after re-training. This could be because the scenarios of the problem are too distinct or that the model is not capable of training with the data set for the new scenario. This has contributed to the statistical recognition of similarities between problem scenarios to recognize the capacity for transfer learning.

Proposed Res-CovNet architectural discussion

At first, the input X-ray image is converted to a wider kernel for the information to be processed with the wider open space. Thus, the input provided for the residual network (ResNet-50) which constitutes of series of convolution layers and pooling layer along with an activation function called, Rectified Linear Unit (ReLu) is instigated for nonlinear batch normalization activation to achieve convergence more rapidly. Due to this, the more variants in the features extracted to process the larger characteristics. Subsequently, the extracted feature set is sent into the following classification layers. The proposed Res-CovNet architecture is a transitional learning model in which the pre-training model is of the residual network (Res Net-50) and followed by the classification layers that will classify the images into one of the four categories as mentioned in Table 1. The architecture of the proposed Res-CovNet can be represented as mentioned in Fig. 4.

Table 1

Dataset distribution

Category	Training set	Validation set
Normal	1108	475
Bacterial pneumonia	1955	838
Viral pneumonia	1036	444
COVID19	105	45
Total	4204	1802

Fig. 4

The proposed system architecture

The workflow on the suggested strategy is demonstrated step-wise in Fig. 4. The similarity level between pneumonia and the COVID-19 is very high when compared with regular pneumonia in the perspective of physiological as well as clinical aspects [10, 29], Transferring expert knowledge from many chest x-ray reports were collected regularly together with some other normal pneumonia individuals might be a good way to remove additional features from small COVID-19 x-ray reports. Subsequently, in the initial testing process, a broader X-ray archive is used for preparing the new Res-CovNet, which is obtained routinely in combination with many other non-COVID individuals with viral as well as bacterial pneumonia. -Below, different input X-ray reports are used to train distinguishable Res-CovNet frameworks individually through ResNet-50 after pre-processing. Later, enhancement of the predictions of the utilized network can be made through the additional classification layers added for the pre-training model, ResNet-50. With the enhancement of the convolution layers to extract the important spatial features from X-ray reports, the weight of these values is transferred immediately in the transition mastering stage. A smaller archive is also used for the instruction of additional layers tuning finely integrated into Res-CovNet for other individuals with pneumonia and COVID-19. Lastly, in the phase of the testing, the improved, stacked, and trained Res-CovNet is utilized to estimate the class of the X-ray image efficiently.

Identification of COVID-19 based on IoMT framework

A hybrid system was generated for the identification of COVID-19, it was a combined framework of the internet of medical things (IoMT) and deep learning-based transfer learning was generated and used for the development of the application and further processing. Architecturally, this system can be divided into two different sections. The first section deals with the deployment of service in the form of a website to consider the input images through the user’s device. The website was developed by using the.NET framework along with the C# language. The input images are further transferred into the processing unit which is an AI-based cloud platform developed by Google that utilizes TensorFlow. The input data dealt with in this system are unstructured, such data effectively handled by MongoDB. Along with that data, information related to algorithm extraction and classification will also be stored using MongoDB. To transfer a message from one device to another medium, the SOAP protocol was used as the part of web service and web-based description language explains the service.

The second section deals with the processing of chest X-ray images, and identification and classification of various forms of pneumonia or COVID-19. This framework was generated by utilizing python. These two sections will go hand in hand and effective output will be generated. Preprocessing is done on the input chest X-ray images while the triggered.NET framework performs a call, which eliminates the anomalies from the image with the aid of methodologies such as histogram equalization or normalization by which image segment features are utilized as input to the various classifiers triggered by the.NET framework. The developed framework preserves the various image formats such as RGB or grayscale or binary or HSV for preprocessing and those obtained images will be passed onto a pre-trained model for training as mentioned in Sect. 3.3 for further evaluation aspects. With the aid of active service pages, the preserved data in MongoDB can be retrieved as well as accessed.

Algorithm and flowchart of the proposed framework

The flowchart of the above-mentioned algorithm based on the proposed framework can be represented as mentioned in Fig. 5.

Fig. 5

Flowchart representation of the working methodology

Results and discussion

This section addresses the dataset used by the database, which is considered to be a feature of IoMT for the proposed system. The findings obtained from the proposed method for the identification of viral pneumonia, bacterial pneumonia, and COVID-19 cases are then discussed. In addition, using a bar chart, this section addresses the distribution of evaluation metrics through different classes of models.

Dataset and system description

The study's dataset came from the Guangzhou Women and Children’s Medical Center in China. There were different categories in the dataset, such as normal and pneumonia. Bacterial pneumonia and viral pneumonia were both included in the pneumonia group. In the dataset under consideration, there are 1583 normal instances. The number of people affected by pneumonia, which involves both bacterial and viral pneumonia, is 4273. Again, there are 2793 cases of bacterial pneumonia and 1480 cases of virus pneumonia. Both of these conditions are known as pneumonia. A separate COVID dataset, consisting of 150 images, was also privately collected and used after verification from a radiologist. As a result, the total data consists of 6006 records in all of the different categories. Each of these classes is further subdivided into two groups: training and validation. 70% of each category's data is in the training collection, while 30% of each category's data is in the validation set. As a result, the training and validation sets each have 4,204 and 1802 members, respectively. As shown in Table 1, the dataset was divided into two parts: the training dataset and the validation dataset. For generating high-performance training and validation of the proposed model is done with high configuration architecture which is comprised of Intel® Core™ i5-2450 M with 2.50 GHz CPU, 64-bit Windows 10 Operating System, and 16 GB of RAM.

The proposed Res-CovNet utilizes images of different resolutions which indicates higher generalizability and can be operated well with knowledge of compact scale. During the transmission analysis process, COVID-19 X-rays worked with various types of conventional pneumonia by tunning the additional layers finely. The predictions generated from the proposed Res-CovNet are optimized with the help of the Transfer Learning Technique for different X-ray images. The combination of ResNet-50 with additional classification layers COVID-19 improves performance in various classifications. At the first stage, the system proposed with ResNet-50 will be pre-tuned to extract features from the various obtained X-ray images as part of training the model. In the later stages, the testing dataset was applied to the trained model to identify the efficiency of the model.

The training dataset was utilized for pre-training using ResNet-50 for the extraction of feature set F. In the later stage, the feature set F will act as input for the additional classification layer. Once training was completed then the later stages will be continued further with testing and validation phases. For the training purpose, the hyperparameters utilized were: learning rate = 0.01, amount of epochs = 24, factor = 0.7, batch size = 10, patience = 5, and drop_out = 0.4. The planned network framework, Res-CovNet was created and assessed utilizing the popular python library, Keras, a rich mastering library with a TensorFlow backend.

Results discussion

The proposed framework results were designed by framing about 5 different models as mentioned in Table 2. Model-1 explains the identification of COVID cases from the normal cases, Model-2 explains the identification of COVID cases from the viral pneumonia cases, Model-3 explains the identification of COVID cases from the bacterial pneumonia cases, Model-4 explains the identification of COVID cases from the viral and bacterial pneumonia cases, and Model-5 explains the identification of COVID cases from all other remaining cases For evaluation of the proposed framework can be done by using various statistical evaluation metrics of the model that are utilized such as accuracy, precision, recall, and F1 score. The confusion matrix plays a vital role in determining the various metrics such as accuracy, precision, recall, and F1-score. The confusion matrix consists of four categories such as True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN). The utilized metrics can be represented mathematically as mentioned in Eqs. 11, 12, 13, and 14.

Table 2

Comparison of the evaluation metrics of the proposed methodology across the various models

Model	Kind of comparison	Accuracy	Precision	Recall	F1-score
Model-1	Normal versus COVID	98.4	97.3	97.7	98.1
Model-2	Viral pneumonia versus COVID	89.3	90.6	88.4	89.7
Model-3	Bacterial pneumonia versus COVID	91.3	93.4	90.6	93.5
Model-4	Viral pneumonia versus bacterial pneumonia versus COVID	92.5	92.2	89.8	89.7
Model-5	Normal versus viral pneumonia versus bacterial pneumonia versus COVID	96.2	90.6	93.4	94.9

The comparison of the evaluation metrics obtained for various models is represented as mentioned in Fig. 6.

Fig. 6

Comparison of evaluation metrics across all the models for the proposed methodology

The proposed methodology proved successful in generating the multi-class classification of various classes such as Normal case, Viral Pneumonia case, Bacterial Pneumonia case, and COVID-19 case. This methodology utilizes chest x-ray images for the recognition of the state of the patient. In the considered database, a greater number of viral and bacterial pneumonia x-ray images are higher than the COVID-19 x-ray images. Due to that uncontrolled biasedness exists while training and testing the model. This condition will get improved as the number of images related to COVID-19 improves in the database. One major challenge in this proposed methodology is to get the data automatically using IoMHT in specific regions around the world. This limitation is only limited to the collection of x-ray images in the database.

Conclusion

The importance of the IoMT frameworks has been increasing day by day. It is highly necessary to incorporate the concept of IoMT with other applications that would be beneficial to society as well. A significant neural framework that forms the core of the processing unit of the developed framework, namely Res-CovNet, presented to effectively identify the various categories such as COVID-19 and pneumonia types using the chest X-ray reports with the concept of transfer learning. This framework was made possible by utilizing the conventional convolutional neural network technique such as ResNet-50 for pre-training the model. As a result, the cost of training decreases, and the extracted features from this phase can be used to evaluate and validate the framework's approach. Initial training of the complete framework, a larger data warehouse is utilized that comprises various categories of X-ray reports such as COVID-19, viral pneumonia, bacterial pneumonia, and normal cases. Because of significant conflicting features among COVID-19 and viral/bacterial pneumonia, a very optimistic solution is achieved by using a smaller repository with COVID-19 X-ray reports by transferring the initially qualified convolution layers with a certain extra tuning of layers finely. The combination of ResNet-50 and additional classification layers is also considered to provide enhanced performance improvements by refining the output estimates for the identification of COVID cases. The framework’s efficiency could be further improved by adding more COVID-19 patient’s sample X-ray reports for guidance in the transfer learning phase. Current outcomes from rigorous studies indicate that other patients with pneumonia and COVID-19 can successfully diagnose other patients even quicker. The Res-CovNet proposed, on the other hand, is highly scalable with broad flexible capacities and can be used in a number of other computer vision applications. This framework can also be used to treat a variety of other deadly diseases that are linked to cancer. Future work on the proposed framework will be focused on this direction.