Internet of Medical Things-Based COVID-19 Detection in CT Images Fused with Fuzzy Ensemble and Transfer Learning Models.

One of the most difficult research areas in today's healthcare industry to combat the coronavirus pandemic is accurate COVID-19 detection. Because of its low infection miss rate and high sensitivity, chest computed tomography (CT) imaging has been recommended as a viable technique for COVID-19 diagnosis in a number of recent clinical investigations. This article presents an Internet of Medical Things (IoMT)-based platform for improving and speeding up COVID-19 identification. Clinical devices are connected to network resources in the suggested IoMT platform using cloud computing. The method enables patients and healthcare experts to work together in real time to diagnose and treat COVID-19, potentially saving time and effort for both patients and physicians. In this paper, we introduce a technique for classifying chest CT scan images into COVID, pneumonia, and normal classes that use a Sugeno fuzzy integral ensemble across three transfer learning models, namely SqueezeNet, DenseNet-201, and MobileNetV2. The suggested fuzzy ensemble techniques outperform each individual transfer learning methodology as well as trainable ensemble strategies in terms of accuracy. The suggested MobileNetV2 fused with Sugeno fuzzy integral ensemble model has a 99.15% accuracy rate. In the present research, this framework was utilized to identify COVID-19, but it may also be implemented and used for medical imaging analyses of other disorders. © Ohmsha, Ltd. and Springer Japan KK, part of Springer Nature 2022.

Introduction

In several countries, IoMT has been used in parallel with other techniques to restrict the spread of COVID-19, enhance the safety of front-line staff, placing greater emphasis by lowering the impact of the illness on human lives, and reduce fatality rates. Specifically, the extensive adoption of IoMT in healthcare facilities may assist in gathering huge medical and healthcare data, which can be utilized by medical practitioners to detect and identify disorders, subsequently proposing suitable therapies. Patients may extend their health status to an IoMT ecosystem through the Internet on a regular basis, and the data is shared with nearby clinics and municipal health agencies [1]. Hospitals could provide eHealth therapies based on the client's health status, and the authorities should provide equipment and allocate quarantine venues such as theaters and hotels. Users may monitor their illness severity and get appropriate medical care with the help of IoMT platform's deployment. It reduces national health expenditures, relieves strain on medical equipment, and provides a complete database that allows the government to avoid disease spread, allocate resources, and implement rapid regulation [2].

This research presents an IoMT-based platform to allow a speedy and safe detection of COVID-19. The potential of IoMT includes more accurate diagnosis, less errors, and reduced costs of treatment. Paired with smartphone apps, the technology enables users to communicate their healthcare information to physicians to better monitor diseases and track and avoid chronic illnesses [3]. This paper describes a COVID-19 diagnostic system based on IoMT that uses several transfer learning methods. First, data from patients is collected via IoT devices and then the sent to the cloud. Image augmentation and preprocessing are done initially once the data has been collected by IoMT devices and transferred to the cloud.

COVID-19 has spread over the globe, and the identification of COVID-19 using CT scans of the lungs has been clinically confirmed. Due to its capacity to detect lung structures, radiological imaging using computed tomography (CT) has emerged as a possible alternative approach of diagnosis. According to relevant research [4], CT scans of the lungs may play a role in the early diagnosis of COVID-19. CT image interpretation and diagnosis is a very complex procedure that requires physicians' professional expertise and experience. The physician's manual experience is a labor-intensive and time-consuming technique. Because RT-PCR test has a poor sensitivity and a high false-negative rate [5], many COVID-19-positive individuals are mistakenly identified as negative. Several deep learning-based techniques for automating the process of identifying COVID-19 infection from lung CT scan pictures have been created recently. However, for the ultimate judgment, the majority of them depend on a single model forecast; this may or may not be right. We employed the Sugeno fuzzy integral technique in this article, which combines the strength of multiple transfer learning models before making a final judgment. Using lung CT scan pictures, we employ several transfer learning models such as SqueezeNet, DenseNet-201, and MobileNetV2. All of these trained models are then combined to form a powerful ensemble classifier, which provides the final prediction.

The article is organized as follows. The Sect. 2 provides a literature review, Sect. 3 discusses the description of the materials and methods in the subsequent Sect. 4 “Result Analysis” contains the experimental data and analyses. The conclusion and future efforts are all mentioned in the concluding Sect. 5.

Literature Review

In the majority of investigations, CT scans were used as imaging modalities for COVID-19 diagnosis. In this sense, the study comprises a number of studies on different transfer learning algorithms for detecting coronavirus using CT imaging. InceptionNet was used by Wang et al. [6] to detect abnormalities linked with COVID-19 in lung CT scan images. InspectionNet was tested on 1065 CT images and found 325 COVID-19-positive patients with an accuracy of 85.20%. Zhao et al. [7] developed a dataset called COVID-CT consisting of 397 and 349 COVID-19-positive and -negative CT scans available for academic usage. Using 3D CT scans, Zheng et al. [8] proposed a poorly supervised DL technique for diagnosing COVID-19 patients. They used a pre-trained UNet technique to segment 3D lung pictures and achieved a 95.9% accuracy rate. In CT scan imaging, Xu et al. [9] used 3D CNN models to detect coronavirus infection from influenza. The author employed the ResNet CNN model and achieved an accuracy of 86.70%. Chen et al. [10] used the UNet design to find coronavirus pneumonia. The authors trained their model on 106 examples and achieved a classification accuracy of 98.85%. COVID-19 detection neural network (COVNet) was developed by Li et al. [11] to recover features from chest CT scans for detection of coronavirus infection in patients, with a 95% accuracy. Angelov et al. [12] achieved 94.96% accuracy on a SAR-CoV-2 CT scan dataset using VGG16, but only for models trained upon unaugmented imagery. Shah et al. [13] employed VGG-19 model and attained an accuracy of 94.52%. Perumal et al. [14] suggested AlexNet paired with SVM model to identify COVID-19 using chest CT scan pictures with an accuracy of 96.69%. Further, a few research [15–36, 46] employed multiple transfer learning models to identify COVID-19 patients.

According to a study of related literature, few researchers have worked on fuzzy ensemble techniques linked with transfer learning models. The fundamental benefit of ensemble learning is that it evaluates and integrates all the choices by multiple models rather than depending on a single classifier [37]. An ensemble will be successful only if individual classifiers exhibit diversity while producing the prediction. Merging deep transfer learning models with fuzzy ensemble techniques may boost the accuracy and durability of a detection system. Our suggested work's originality and key contributions are as follows.

The current study leverages the ensemble learning approach for classifier fusion.

We have used SqueezeNet, DenseNet-201, and MobileNetV2 transfer learning models as foundation models. To aggregate the basic models’ predictions and to provide superior outcomes, we employed trainable ensemble and Sugeno fuzzy integral ensemble techniques.

In this work, the Sugeno fuzzy integral was used to assemble the aforesaid classifiers to overcome the drawbacks of using the simple fusion technique.

Fuzzy integrals [38] are efficient aggregators that use the level of unpredictability in evaluation scores as extra info for classifier fusion. It may be considered of as a generalization of aggregation procedures on a collection of confidence scores with some weighting assigned to each source of data, referred to as fuzzy measures. Unlike the fundamental fusion techniques in the literature, which use fixed pre-determined weights, the Sugeno fuzzy integral uses the confidence of base learners' predictions to add adaptive weights to each input to the model.

Materials and Methods

Proposed Framework

Figure 1 depicts the suggested IoMT-based framework for COVID-19 classification from CT images. Smart IoT sensors are used to send CT scan images. Low-range networking equipment constitutes the local area network (LAN). From the intelligent IoT sensor and the device, this layer communicates the obtained signals to the next layer called the hosting layer. Various intelligent devices, such as portable multimedia or computers that can store and transfer signals, constitute the hosting layer. The intelligent devices are linked to a wide area network (WAN), which sends data from the devices to the cloud. The WAN layer sends information to the cloud in real time via specialized networks such as Cellular LAN, 4G, or 5G. Patient information is authenticated and sent to the transfer learning cognitive module via the cloud manager. Data is sent through intelligent IoT sensors. Any of these sensors may be included into the patient's environment. This gadget can also communicate with other IoT devices. The LAN is made up of short-range networking technologies such as Bluetooth and Zigbee.

Fig. 1

Proposed IoMT architecture for COVID-19 detection

The hosting layer supports smart devices such as tablets, mobiles, human digital assistants, and laptops. These gadgets have specific programs that compute the signals received and save data locally. These compact processing tools allow users to acquire generic and provisional health assessments. The data is sent to the cloud processing unit over the WAN layer. A cloud manager and a transfer learning cognitive module make up the cloud layer. The cloud manager is in charge of data flow and ensures that all authentication mechanisms are in place to ensure that all intelligent city actors' identities are verified. After patient verification, the transfer learning cognitive module analyzes the data and evaluates the patient's condition. It makes informed judgments based on CT scan images to detect COVID-19. Finally, medical specialists will analyze the data and keep track of the patients. Potential care may be assessed if the patient requires emergency treatment.

The cognitive system checks the patient's state and sends the CT scan image to the cloud, where the transfer learning cognitive module may evaluate it. Using chest CT scan images, the suggested automated framework achieves excellent classification sensitivity and accuracy while also being a considerably quicker technique. The suggested technique may be utilized with fresh test pictures being sent through the model to provide ensemble predictions. The complementary nature of the data obtained by the multiple transfer learning models is proven by examining the statistical dissimilarities from the decision values for each transfer learning model. The transfer learning classifiers are combined using the trainable ensemble and the Sugeno fuzzy integral. To extract features and classify them accurately, the CT scan images in this study must be suitable with the pre-trained transfer learning models. As a result, the first step is to scale the images to their original proportions, which is the standard form employed in neural networks. The COVID-19 is detected by the transfer learning module, which then returns the three-classification effects. The cognitive system anticipates future tasks based on these discoveries. These findings are shared with health specialists in the form of medical reports for a full examination. In the case of an emergency, the cognitive gadget generates alerts and messages, and a smart ambulance can quickly identify and meet the patient. The smart traffic technology also enables emergency vehicles to get to their destination in the quickest possible time by using the shortest route. Transfer learning models fused with fuzzy ensemble technique cognitive module is depicted in Fig. 2. A 224 × 224 × 3 input image was used, as well as a dense layer with 4096 nodes and a softmax layer with three output nodes.

Fig. 2

Transfer learning models fused with fuzzy ensemble techniques cognitive module

Transfer Learning Models

SqueezeNet SqueezeNet is a CNN that uses design tactics to minimize the number of parameters, particularly via the use of fire modules, which squeeze parameters using 1 × 1 convolutions. It is an 18-layer deep CNN. With 50 times less parameters, it can attain an accuracy equivalent to AlexNet [39]. SqueezeNet's Fire module is a critical component for efficiently reducing the amount of parameters. SqueezeNet starts with a single convolution layer, then eight fire modules, and a final convolution layer. From the beginning to the conclusion of the network, we progressively expand the number of filters per fire module. After layers conv1, fire4, fire8, and conv10, SqueezeNet executes max-pooling with a stride of 2; these pooling locations are rather late (Fig. 3).

Fig. 3

SqueezeNet

DenseNet-201 The DenseNet-201 is a 201-layer CNN. DenseNet201 utilizes the condensed network, giving simple to train and extremely parameterized effective methods [40]. This is due to the possibility for feature reusing by multiple layers, which enhances variance in the subsequent layer input and improves performance. DenseNet201 has performed well on a variety of datasets, including ImageNet and CIFAR-100. In DenseNet-201, the network links each layer to the next in a feed-forward fashion. Each subsequent layer uses the feature maps from the previous levels as input. Each layer in this network receives a cumulative knowledge from all the levels before it. Every preceding layer's connection is added to the DenseNet-201. The DenseNet has the benefit of reducing the disappearing issue since it incorporates feature maps from all previous levels. It improves feature propagation while reducing the number of parameters (Fig. 4).

Fig. 4

DenseNet201

MobileNetV2 One of the most lightweight network architectures is MobileNetV2. The inverted residual with linear bottleneck layer module in MobileNetV2 greatly minimizes the amount of memory required for processing. MobileNetV2 expands on MobileNetV1's concepts by using depth-wise separable convolution as an efficient building element. As shown in Fig. 1, the MobileNetV2 design [41] has two kinds of blocks: a residual block with a stride of 1 and a shrinking block with a stride of 2. Both kinds of blocks feature three layers, with the first layer including an 11 convolution with ReLU6, the second layer containing depth-wise convolution, and the third layer containing another 11 convolution without non-linearity. Figure 4 depicts the input, operator, and output of each layer (Fig. 5).

Fig. 5

MobileNetV2

Data Set

The China National Center for Bio-information provided the CT chest imaging dataset that we utilized in our investigation. Three groups of photos have been labeled. The collection contains CT images of 999 COVID-19 patients, 1468 pneumonia patients, and 1,687 healthy individuals. From the whole dataset, we utilized an equal number of pneumonia, COVID-19-positive, and normal CT scans. There are 650 images in each class. The CT image provided to the system is 224 × 224 × 3 pixels in size. To assess whether or not a person has breast cancer, Tensor Flow and Keras deep learning models were developed. The data was split into 70% and 30% ratio for the training and test set, with the same groups being used for all models. Few pre-trained model layers were utilized to extract the characteristics from the training images. The pre-trained models were able to classify lung CT images based on class labels given to the training dataset. In this paper, we have used the balance dataset and the learning curve is shown in Fig. 6.

Fig. 6

Learning curve

Experimental Environment

The suggested architecture was developed in Python 3.6 with PyCharm in a Windows 10 environment, employing several AI and image-processing packages to improve training efficiency and speed. Numpy, scipy, openCV, and fastai libraries were used in our testing setup, which was accelerated by a GPU with 8 GB dedicated memory running the Keras deep learning framework backend. We utilized 25 epochs and a learning rate of 0.0001 for training, which were both small enough to prevent overfitting the transfer learning models. We utilized the SDG optimizer for compilation, and after extracting features from pre-trained models, employed dense layers with 4096 neurons each as part of the classifier, using Rectified Linear Unit (ReLU) as the activation function. The accuracy curves produced by the three models after training are presented in Fig. 7.

Fig. 7

Accuracy graph for a SqueezeNet b DenseNet-201, and c MobileNetV2

Ensemble Technologies

Ensemble learning is a method of combining the most important characteristics of two or more base learners. Because ensembling minimizes the variation in the prediction errors, such a framework performs better than its component models.

Trainable ensemble In this scenario, we utilized a different classifier to combine the scores of all the basic models. All of the basis models' class scores were sample-wise flattened into a single feature vector. Then we chose a classifier that provided excellent results when we compared the flattened test and train data scores to the outcomes of the train and test. Here, we used a multilayer perceptron (MLP) in our research. Nodes are the primary components of a neural network (NN). These nodes are frequently grouped together as layers. In NN, data is sent from one layer to the next. The flow of information in the feed-forward neural network (FNN) [42] is fixed in one direction, from the input layer to the output layer. Assume the preceding layer has nodes, with each node forwarding the value to a specific node in the current layer. The output of node will then bewhere is the weight allocated to the route from node to node , and is the activation function present in the current layer. The basic goal of FNNs is to improve .

The FNN must include at least one hidden layer between the input and output layers in any MLP [43] design. We used a basic MLP architecture with just one hidden layer and 16 features in our research. The MLP took a total of score features as inputs and returned score values for all classes, allowing the ensemble model to utilize the deep learning network as an aggregator.

Sugeno fuzzy integral Tahani et al. [44] introduced the Sugeno fuzzy-λ measures. Assume an accumulation of decision scores where is the number of information sources (in this example, M = 3). The function is the Sugeno-λ measure. is a number that spans from 0 to 1 and meets the following criteria:

1.

If e , Eq. (1) holds truth value if and only if

As a result, as shown in Eq. (2), λ is the real root.

The Sugeno integral [45] definition is stated as: If () is a measurable (Borel) space, is a μ-measurable function, Eq. (3) shows the Sugeno integral of the measurable function D in terms of fuzzy measure ψ.

1 ≤ i ≤ n, where and are the ranges specified as . The detailed Sugeno integral algorithm is mentioned below.

Result Analysis

The fuzzy logic-based ensemble works particularly well when assigning weights to the predictions for reaching an ultimate decision on an image's class since the confidence in a classifier's prediction is taken into consideration for each sample when allocating weights to the predictions. Table 1 shows the results of an ensemble formed using three transfer learning models, demonstrating that the Sugeno integral greatly outperforms the others. The trainable ensemble approach is also very effective. Each model's accuracy, F1-score, sensitivity, and specificity were assessed, with the results shown in Table 1. Figures 8, 9, and 10 demonstrate the confusion matrices of the SqueezeNet, DenseNet-201, and MobileNetV2 using fuzzy ensemble approaches. Table 2 compares the results of several transfer learning methodologies to the suggested CT imaging methodology.

Table 1

Performance indicators for various transfer learning models using fuzzy ensemble technologies

Models	Class	Recall	Precision	Specificity	F1-score	Accuracy (%)
SqueezeNet	Positive	0.964	0.974	0.987	0.969	97.09
	Normal	0.979	0.965	0.982	0.972
	Pneumonia	0.969	0.974	0.987	0.972
SqueezeNet + trainable ensemble	Positive	0.969	0.984	0.992	0.977	97.44
	Normal	0.979	0.970	0.985	0.974
	Pneumonia	0.974	0.969	0.985	0.972
SqueezeNet + Sugeno fuzzy integral	Positive	0.974	0.979	0.990	0.977	97.78
	Normal	0.979	0.979	0.990	0.979
	Pneumonia	0.979	0.974	0.987	0.977
DenseNet-201	Positive	0.974	0.974	0.987	0.974	97.61
	Normal	0.979	0.979	0.990	0.979
	Pneumonia	0.974	0.974	0.987	0.974
DenseNet-201 + trainable ensemble	Positive	0.979	0.970	0.985	0.974	97.95
	Normal	0.979	0.985	0.992	0.982
	Pneumonia	0.979	0.985	0.992	0.982
DenseNet-201 + Sugeno fuzzy integral	Positive	0.985	0.980	0.990	0.982	98.29
	Normal	0.979	0.985	0.992	0.982
	Pneumonia	0.985	0.985	0.992	0.985
MobileNetV2	Positive	0.985	0.975	0.987	0.980	98.12
	Normal	0.979	0.990	0.995	0.985
	Pneumonia	0.979	0.979	0.990	0.979
MobileNetV2 + trainable ensemble	Positive	0.985	0.990	0.995	0.987	98.80
	Normal	0.990	0.985	0.992	0.987
	Pneumonia	0.990	0.990	0.995	0.990
MobileNetV2 + Sugeno fuzzy integral	Positive	0.990	0.990	0.995	0.990	99.15
	Normal	0.990	0.990	0.995	0.990
	Pneumonia	0.995	0.995	0.997	0.995

Fig. 8

SqueezeNet, SqueezeNet + trainable ensemble, and SqueezeNet + Sugeno fuzzy integral confusion matrix representation

Fig. 9

DenseNet-201, DenseNet-201 + trainable ensemblec and DenseNet-201 + Sugeno fuzzy integral confusion matrix representation

Fig. 10

MobileNetV2, MobileNetV2 + trainable ensemble, and MobileNetV2 + Sugeno fuzzy integral confusion matrix representation

Table 2

The comparison of the performance of several transfer learning models with the suggested models for COVID-19 detection using CT imaging

Authors	Technology	Accuracy (%)
Wang et al. [6]	InceptionNet	85.20
Polsinelli et al. [17]	Light SqueezeNet CNN	85.03
Santa et al. [16]	Two-stage transfer with stacked ensemble	86.70
Angelov et al. [12]	VGG16	94.96
Shah et al. [13]	VGG-19	94.52
Yu et al. [18]	DenseNet-201 with cubic SVM model	95.34
Zheng et al. [8]	UNet technique	95.9
Perumal et al. [14]	AlexNet combined with SVM model	96.69
Krishnaswamy et al. [20]	Fused lightweight CNN model	97
Halder et al. [15]	DenseNet201	97
Yan et al. [19]	AI-based multi-scale convolutional neural network	97.7
Chen et al. [10]	UNetþþ architecture	98.85
Alquzi et al. [21]	EfficientNet-B3 CNN	99
Proposed model	MobileNetV2 fused with Sugeno fuzzy integral	99.15

Conclusion

For COVID-19 patient recognition using CT scans, this research provides an IoMT-based technique combined with fuzzy ensemble and transfer learning models. Early identification of pneumonia is critical for deciding the best course of therapy and preventing the illness from posing a life-threatening hazard to the patient. This work created an IoMT-based system that combines deep transfer learning models combined with a fuzzy ensemble technique to categorize CT images into three classes: normal, COVID-19 positive and pneumonia. The suggested approach assembles the characteristics of three pre-trained models using deep transfer learning methods and Sugeno integral. The accuracy rate of the recommended MobileNetV2 fused with trainable ensemble and Sugeno fuzzy ensemble model is 98.80% and 99.15%, respectively. In the future, this research might be expanded to enable the identification of a range of lung infections using CT scan image. The suggested model may be modified to include picture fusion methods in future study. The implementation of IoMT may alleviate the constraints placed on healthcare systems, yet IoMT also has security and privacy flaws. Blockchain technologies, on the other hand, have the potential to improve the privacy and security of IoMT systems.