Remote diagnostic and detection of coronavirus disease (COVID-19) system based on intelligent healthcare and internet of things.

In this paper, we will propose a novel system for remote detecting COVID-19 patients based on artificial intelligence technology and internet of things (IoT) in order to stop the virus spreading at an early stage. In this work, we will focus on connecting several sensors to work together as a system that can discover people infected with the Coronavirus remotely, this will reduce the spread of the disease. The proposed system consists of several devices called smart medical sensors such as: pulse, thermal monitoring, and blood sensors. The system is working sequentially starting by pulse sensor and end by blood sensor including an algorithm to manage the data given from sensors. The pulse sensor is devoted to acquire a high quality data using a smartphone equipped by a mobile dermatoscope with 20× magnification. The processing is used RGB color system to perform moving window to segment regions of interest (ROIs) as inputs of the heart rate estimation algorithm. The heart rate (HR) estimation is then given by computing the dominant frequency by identifying the most prominent peak of the discrete Fourier transform (DFT) technique. The thermal monitoring is used for fever detection using a smart camera that can provide an optimum solution for fever detection. The infrared sensor can quickly measure surface temperature without making any contact with a person's skin. A blood sensor is used to measure percentages of white, red blood (WBCs, RBCs) volume and platelets non-invasively using the bioimpedance analysis and independent component analysis (ICA). The proposed sensor consists of two electrodes which can be used to send the current to the earlobe and measure the produced voltage. A mathematical model was modified to describe the impedance of earlobe in different frequencies (i.e., low, medium, and high). The COMSOL model is used to simulate blood electrical properties and frequencies to measure WBCs, RBCs and Platelets volume. These devices are collected to work automatically without user interaction for remote checking the coronavirus patients. The proposed system is experimented by six examples to prove its applicability and efficiency.

Introduction

Medical Technologies could be improved for the future of the health care to deliver health services at home or workplace [1], [2]. Sensors could be available in the form of accessories, or they could be integrated into clothing and the environment. They’re used in collecting vital human or environmental health information in daily life [3], [4]. In addition, sensors can even be designed to be skin-on-chip to allow long-term health monitoring [5]. Cloud storage, on the other hand, is a type of Internet-based service where stored data is provided and shared with computers and other devices on-demand. Therefore, Sensors generate and collect various health bio signal streams that are analyzed locally to build a user profile and transmitted wirelessly through mobile or web applications to private cloud storage [6]. Medical data can be measured with sensors in two ways; invasive and non- invasive. The probe is entered to the human body through the natural cavities of the gills, throat, and ears in case of invasive sensors. Therefore, the doctors always prefer non- invasive sensors to avoid the side effects as a result of surgery [7].

There are several non-invasive techniques can be used in blood analysis, such as optical spectroscopy and fluorescence technology, etc. These techniques had various limitations in sensitive to motion artifacts, interferences, skin temperature, and low signal to noise ratio. Therefore extensive processing of data [8] is needed to check any harmful artifacts such as viruses, bacteria, blood diseases and etc. that afflict humans. Recently, the whole world has become interested in finding a way for detection the Corona virus, so that it can get treatment early. Symptoms of the Corona virus include a fever, a high heart rate, and an imbalance of red and white blood cells. Therefore, emotions analysis and management represent the most attractive side of recent emerging of Internet of Things (IoT) applications and smart wearable sensors [9]. Recently, the IOT has demonstrated great potential for qualifying and improving health care services such as health monitoring at anytime and anywhere. This field has provided innovative new methods through which infectious diseases can be detected and diagnosed remotely using smart sensors. The data generated from these sensors can be processed and stored or broadcast to a remote device (such as a cloud server). IoMT, on the other hand, offers the option of connecting embedded medical sensors, actuators, or other devices to the Internet. Therefore, the vision of a global infrastructure of networked physical objects is considered to be an enabling technology [10]. In the future, it necessary to rely on embedded medical sensor based on information technology, artificial intelligence and the Internet of things to diagnose dangerous viruses such as Corona [11], [12]. There are also many continuous efforts and a lot of research to mitigate the spread of this virus, but the internet of things is considered one of the safe and effective ways to deal with the Corona pandemic [13], [14]. Until now, I have no knowledge of any published methods that use the internet of things to discover or examine a Corona patient, so that we can rely on them as reference results.

In this paper, a novel system is proposed for remote COVID-19 patient’s detection based on intelligence technology and internet of things (IoT). The proposed system composes of three smart medical sensors called pulse, thermal monitoring, and blood sensors. We concentrate on mathematical representations to estimate the heart rate (HR), fever degree, and pulse blood in order to show the changes of the vital functions of the patient’s body. The pulse sensor is used to compute the HR using set of video frames come from a mobile dermatoscope with 20× magnification. The graph in a frame is transformed to peaks using fast Fourier transform (FFT). The FFT curve is analyzed to different regions of interest (ROIs) to estimate the heart rate. In case of thermal monitoring sensor, the infrared sensor can quickly measure surface temperature without making any contact with a person’s skin. Then, a blood sensor is proposed to measure white blood (WBCs) and red blood (RBCs), Platelets volume. In this sensor, we use the two electrodes to send the current to the earlobe and measure the produced voltage. The COMSOL model is used to measure WBCs, RBCs and Platelets volume based on blood electrical properties and frequencies. For that, we modified a mathematical model to describe the impedance of earlobe in different frequencies (i.e., low, medium, and high). Six examples are given to prove the efficiency of the proposed system according to manual investigations.

The research can be described as follows: In Section “The proposed medical sensor system”, a novel medical sensor is proposed. We present the architecture of the system in Section “The structure of the proposed system”. In Section “Experimental results”, the experimental results are given. We conclude the work in Section “Conclusion”.

The proposed medical sensor system

In this section, we will state the structure of the proposed medical system which is effective to COVID-19 detection based on intelligent technology and IoT. The proposed system is composed of the three sensors: pulse, thermal, and blood sensors. This combination of sensors will help us to get an early detection of potential COVID-19 or feverish patients. A mathematical model has been used to describe the transmitted light oximeter and reflected oximeter as in pulse sensor. We also improve the detection of the time series signal by ignoring effects of motion and concentrating on the region of interest (ROI) locations. We suppose that the ROI is varied in red, green and blue (RGB) instead of one color as in [15]. For thermal sensor, we used a system for fast noncontact screening for fever human subjects [16]. This camera provided optimum solution for fever detection which gives the opportunity to stop the infection from spreading. The blood sensor has been provided to measure white blood cells (WBCs) and red blood cells (RBCs) levels non-invasive using the bioimpedance analysis with respect to the change of blood electrical properties.

Pulse sensor

The pulse can be estimated by calculating the frequency data of the alternating current (AC) portion of heart rate (HR). Non-contact estimation offers a less accurate way to estimate HR [5]. This involves the smartphone application that operates under perfect illumination conditions and the application of blind source separation (BSS) based on component analysis (ICA) technique. HR estimation by applying blind source separation (BSS) using the ICA technique, see Ref. [15] for more explanation. In this section, we improve the HR by estimating the time series signal by ignoring effects of motion and concentrating on the region of interest (ROI) locations. The ROI is provided in red, green and blue (RGB) instead of one color as in [15].

The absorption of light by human skin relies on melanin in the epidermis and hemoglobin. We assume that the skin absorbance is linear combination with melanin and hemoglobin absorbance in log space [12], [13], [14]. The skin absorbance B at λ by using Beer’s law can be represented as:where λ, m and h are wavelength, melanin and hemoglobin respectively. c, v and v are the pigment concentration, the product of pigment extinction coefficient and the mean path length of photons in the skin layer respectively. B can be interpreted by equation:where D and L are the power of transmitted light and incident light. From Equations (1), (2), we can get:

We use RGB model to describe skin image locations as (i = R,G,B) at position (x,y).where k is a constant for the gain of the camera and S is the spectral response function for the camera sensor. By neglecting the side lobes and then taking logarithm in both sides, we can yield to:

The Region of Interest (ROI) module is used to separate the object from the source image. The Region of Interest module enables users to define 3-D regions and to measure them. To define regions of interest within a volume image, we use the tools in the module. The pixels at each frame are arranged into trace of three components (R,G,B). The RGB traces are normalized as follows:

We define Q as:

Let where μi and σi are the mean and standard deviation for each i = 1,2,3 corresponding to (i = R,G,B). and are the normalization transforms. We decompose the normalized traces into three independent source signals using ICA.where

We use ΔQ as the subtraction between consecutive frames:t is the frame number and Δch is the blood concentration whose frequency, by definition, is the pulse HR. According to [5], Δ Δ can be interpreted as the AC component while the term Δlog can be interpreted to be the DC component if the illumination is kept constant. The Eq. (8) can be rewritten as:

Assume that the video sequence contain m frames, we can compute the HR by estimating the time series signal y(t) from:

The HR can be estimated counting the most prominent peak of the discrete Fourier transform (DFT) technique.

Thermal sensor

A system for quick real-time monitoring of human fever is called a thermal imaging camera. Dangerous infections (fever) manifest at elevated body temperatures. For instance, SARS is characterized by a high body temperature (HBT) of 38 °C. Body temperature is universally recognized as an important indicator of humans and other warm-blooded animals' physical condition. The thermal imaging camera is paired with a target gate that combines the reference blackbody targets and the position detectors to identify a subject's temperature scale and size. Combined with a target gate to spot the ordered series and size of a subject, it comes with the reference blackbody and position detectors. The snapshot for thermal imaging is managed by the position detectors. An afebrile subject generates thermal image pixels that fit within the predetermined normal temperature range calibrated by the blackbody targets of reference and have no pixels above that range. Exceeding that range by at least two adjacent pixels is a sign of fever that triggers a subject's alarm and size. The position detectors regulate the thermal imaging snapshot. An afebrile subject would produce pixels in a thermal image that fit within the predetermined. The embodiment and the process of noncontact screening for fever human subjects are explained in detail in [16] to show the operations of equipment referencing from 1 to 14 in [16]. In our experiments, two thermometers are used RC5 and DHT22 to measure air temperature surrounding the smartphones [17]. They should work independently with smartphone. RC5 is considered as the reference thermometer while the DHT22 is used to record ambient air temperature. The smartphone will record the temperature of the smartphone's battery using the Android app, the sensor monitor. Data is collected from thermometers and smartphones, and experiments are conducted to build models of predicting air temperature, see [17] for more explanations.

Blood sensor

Blood consists of four components suspended in plasma which are cholesterol, White Blood Cells (WBCs), Red Blood Cells (RBCs), and Platelets. The analytic process of blood components can be done with taking a sample of the subject’s blood. This method has many disadvantages especially in the current time during the COVID-19 pandemic since it may help with infection transmission. A non-invasive blood analysis method is widely used to avoid this problem relates to Bioelectrical impedance analysis (BIA) which is calculating the impedance value by applying current. We study using different frequencies and their effect on electrical properties of blood components.

It’s known that the high white blood cells is a good indicator and a sign for a disease since white blood cells make up around 0.5% of the total blood volume so any increase to this percentage can be noticeable with any blood components analysis method. COVID-19 like any viral zoonotic diseases can cause high blood cells except in severe cases, but since most of severe cases are apparent and can be detected with thermal monitoring it will not cause an issue. The structure of the system can be described in the following subsections.

Electrical properties

The general law describing the impedance of blood () at various frequencies is demonstrated in Eq.(11), where is the blood volume with the area A between the two electrodes. and are blood permittivity property and the electrical conductivity respectively, and i,j are imaginary values [18].

Now, we have three frequencies, high (HF), medium (MF) and low (LF). We can compute the relationship between blood conductivity and the plasma conductivity of cells and particles in case of low frequency aswhere h is small fraction. By substituting Eq. (12) in Eq.(11), we can get:

To subdivide the blood components into blood cells (red blood cells, white blood cells and platelets) and plasma, we can use Maxwell-Fricke equations to distinguish between plasma containing cholesterol particles and blood cells. For cholesterol particles, we assume the volume, electrical conductivity and electrical permittivity are:, and , and , and for fluid. In the case of high frequencies, Eq. (13) can therefore be changed as follows:

A blood model is used for earlobe in order to detect the blood diseases. This model consists of five layers. The components of layers are: two skin layers, two tissue layers, and blood layer (i.e.; plasma, RBCs, WBCs, and platelets) where each layer has a specific thickness and electrical properties. Let we have, and are electrical conductivity the thickness and electrical permittivity for skin and, and are the thickness, electrical conductivity and electrical permittivity of adipose tissue. Eq. (11) is modified by adding the impedance of each layer.

Measuring red blood cells

Now, a blood sensor is used to measure white blood cells (WBCs) and red blood cells (RBCs) levels non-invasively using the bioimpedance analysis. We simulate the COMSOL MULTIPHYSICS 5.0 for the impedance of earlobe in three different medium frequencies to measure percentages of cells in the blood. The Eq. (14) can be devoted as the following according to the frequencies rate as:where t is for white, red cells and platelets (W, R, and P) corresponding three frequencies (MFl), l = 1,2 and 3 respectively, for low frequency, we noted that . can be obtained by solving these three equations

In COVID-19, we assume that the difference between the measurements of frequencies of red cell and plates are small and yield to zero:

We can get also:

Similar:

Assume that and the change in other cells impedance is neglected compared with the red blood cells.

From Eqs. (17), (18), we get:

We assume:

Then

White blood cells volume can be obtained from Eq. (17):

To measure red blood volume, by neglecting the change of platelets volume during different frequencies, then according to Eq. (16), we can get:

Similar:

Assume that

We assume:

We get:

RBCs volume:

Similar, the Platelets volume can be obtained at follows:

Similar:

Assume that

We assume:

We get:

RBCs volume:

The structure of the proposed system

Now, to recover COVID-19 patients without requiring special treatment, we compose the equipment that can find the HR by estimating the time series signal y(t) as in Eq. (9), estimate a high body temperature (HBT) as thermal sensor and compute blood components (WBCs and RWCs, platelets) using Eq. (19), Eq. (20) and Eq. (21) respectively. The proposed system is designed based on pulse, thermal monitoring, and blood sensors based on artificial intelligence technology and internet of things (IoT) algorithm to earlier detection for coronavirus disease.

The main components of the proposed medical system are:

Sensor electrodes: consists of two electrodes: upper and lower electrodes.

Digital Camera: recording 30 frames per second (24-bit).

Thermal monitoring sensor: characterizing a high body temperature.

A microcontroller: is used for receiving and processing the measured signal from the sensors, determining appropriate responses according to equations, and managing local memory.

. I/O buses: is used transfer analog or digital data in cables between different parts of the system.

Accelerometer: is used to measure the change in the results produced by the sensors

Three smartphones (with 20× magnifications): one is a mobile dermatoscop to estimate the HR using set of video frames. Other smartphone is to receive data from the wireless sensors which used for measuring white blood cells (WBCs) and red blood cells (RBCs) levels. Smartphone is used to estimate air temperature. It uses to estimate a human skin temperature at different positions of the body.

Thermometers: is the equipment DHT22 and RC5. RC5. RC5 is used to measure temperature and will be considered as the reference thermometer. The DHT22 is used to record the ambient air temperature. Smartphone was recorded with the Android app, which is the Sensor Monitor

Memory: is used to store the operating system, application software and data generated from all sensors and all electrophoresis processors.

Power supply: we used battery to supply component in the system.

The proposed algorithm works according to the system sensors results. For instance, the algorithm begins by measuring the patient's temperature using the thermal sensor, i.e. if there is a fever, i.e. the temperature is more than 37 °C. The pulse sensor calculates the heart rate and check it if it is more than 100. Then, the blood sensor works at the counting of white, red blood cells and platelets Hematologic. The output of the proposed algorithm is the patient test is positive and must isolate him for more laboratory tests.

Experimental results

We have presented several experiments to prove the advantages of proposed system. Six examples (A-F) are tested with our proposed system to show its applicability and efficiency. Video sequences acquired from different body parts, where three different cameras are used corresponding proposed sensors, pulse, thermal, and blood. The system is working sequentially starting by pulse sensor and end by blood sensor.

Heart rate estimation

We proposed an algorithm to manage the data given from sensors. For example, the pulse sensor is working by acquiring high quality data using a smartphone equipped with a mobile dermatoscope [18], [19]. The dermatoscope allowed videos to be taken with 20× magnification. The processing is performed using a 30 s moving window with 96.7% overlap (1’s increment) with RGB pixels. For testing the consistency of the proposed algorithm, different regions of interest (ROIs) [18] are selected as inputs to separately estimate the heart rate. The time traces y(t) are computed as in Eq. (9). The corresponding dominant frequency in the Fourier spectrum matches with the pulse oximeter, used to record the heart rate (HR) at the same time, for ex. in case A, heart rate = 1.185 × 60 = 70.98. We noted that the FFTs are exactly identical with accuracy for each trace obtained from same person. Fig. 1 is described the steps of acquiring data using pulse sensor.

Fig. 1

The movie sequences taken with the smartphone dermatoscope and the corresponding time traces y(t) are computed as in Eq. (9).

Statistical analysis

To show the statistical analysis of the system results, several experiments were presented to prove its applicability. We applied the proposed system on six cases. We start by pulse sensor to estimate the heart rate as in next subsection. After that we apply thermal monitoring sensor to get the high body temperature (HBT) as found in [19]. The blood sensor is applied to body location. For the tested examples, the different values of blood electrical properties and frequencies are given. The COMSOL model is applied on blood electrical properties and frequencies of the measured of WBCs, RBCs and Platelets volume to calculate the WBCs, RBCs and Platelets in the blood according to Eqs. (19), (20), (21) as shown in Table 1 . For WBCs, RBCs, and Platelets corresponding three frequencies low, medium, and high respectively. Fig. 2, Fig. 3 represent the variation of electrical properties of blood in different frequencies. Fig. 2 shows an increase in blood permittivity as the frequency increases. It describes the chart of blood electrical conductivity property (θ) for six cases in different frequencies. Fig. 3 shows that blood electrical permittivity) (for six cases in different frequencies. This effect is more significant when the frequency reaches 4 GHz; meanwhile the conductivity property is the highest comparing to other tested frequency when the frequency reaches 300 MHz. It is known that at very high frequencies, the resistive property of an object increases dramatically due to temperature and it’s not advised to reach 1 GHz as the cell membrane can begin to deteriorate and allow the cytosol to leak into the extracellular space and cause harm. Blood is the main vital element in the human body which consists of different cells suspended in plasma (around 55% of blood volume). The components of blood are; 0.5% White Blood Cells (WBCs), 45% Red Blood Cells (RBCs), and 0.5% Platelets [20]. COVID-19 diseases affect the human blood and can change these percentages to be high or low. The number of WBCs, RBCs, cells and Platelets can be computed as follows:

Table 1

The electrical properties of the blood at different frequencies (low, medium, and high).

Examples	Low Frequencies				Medium Frequencies		High Frequencies
	100 kHz		200 KHz		300 MHz		500 MHz		4 GHz
	θ	β	θ	β	θ	β	θ	β	θ	β	RBCs volume	WBCs volume
A	0.3418	43.11	0.3991	38.91	2.002	70.0234	2.192	62.880	8.333	49.310	48	0.55
B	0.2888	39.43	0.311	35.21	1.999	68.999	2.091	64.184	8.411	50.311	44	0.48
C	0.2046	26.29	0.226	23.57	1.999	69.786	2.090	64.003	8.321	50.083	45	0.60
D	0.3871	43.45	0.353	39.95	2.001	69.781	2.092	64.780	8.512	50.120	50	0.58
E	0.2999	39.78	0.351	34.01	2.011	70.321	2.091	63.991	8.444	50.211	51	0.53
F	0.19992	26.19	0.235	23.57	1.993	69.852	2.090	63.893	8.332	49.999	45	0.49

Fig. 2

Comparing blood electrical conductivity property (θ) for six cases in different frequencies.

Fig. 3

Comparing blood electrical permittivity)(for six cases in different frequencies.

The count of platelets is:

For WBCs:

For female RBCs:

For male RBCs:

Comparative results

To show the efficiency of the proposed method, we compute the dispersion to determine the degree to which the data deviates from the mean value of the tested samples by the following equation.where, are the data set that obtained from the proposed system and manual estimation respectively. We compute the error between two data sets as shown in Table 3. For Error = 2.8, this means that RBCs volume estimation of the proposed system can be shifted over the real estimations by 2.8 value, etc. This error is considered acceptable and gives an indication of the efficiency of the system comparing to the manual system that require physical contact, although there is a slight margin of error.

Table 3

The real result for cases A-F based on Table 1 computed from manual estimation.

Examples	RBCs volume%	WBCs Volume%	Platelets Volume%	HBT	HR
A	52	0.53	0.103	37.5	73
B	46	0.55	0.100	37	80
C	45	0.54	0.198	37.6	73
D	53	0.57	0.980	36.9	75
E	55	0.57	0.136	38	80
F	43	0.56	0.106	38.5	77
Error	2.8	0.05	0.045	1.02	3.52

The six outputs are feed to the algorithm to distinguish that if the person with symptoms of COVID-19 or not. Although the proposed system gives initial results, it promises to detect COVID-19 remotely without human intervention. Table 2 shows the provided data (WBCs volume, RBCS volume, Platelets Volume) from the proposed system which proves the efficiency of the proposed algorithm comparing to the manual diagnostic in Table 3 . These results prove that the algorithm is successful and stable for test cases, for ex. In WBCs, 0.55, 0.48, 0.60, 0.58, 53, and 0.49 are the proposed algorithm outputs, these cases are tested using lab analysis and the results were 0.53, 0.55, 0.54, 0.57, 0.57, and 0.56 respectively. Because it is not easy to get the data from more corona patient examinations, many test cases are need for examination in future.

Table 2

The proposed algorithm output for cases A-F based on Table 1 computed from Eqs. (19), (20), (21).

Examples	RBCs volume%	WBCs Volume%	Platelets Volume%	HBT	HR
A	48	0.55	0.100	39.5	72.98
B	44	0.48	0.105	38	83.1
C	45	0.60	0.209	37	76.92
D	50	0.58	0.99	36.9	68.08
E	51	0.53	0.13	38	75.01
F	45	0.49	0.11	38	76.8

Conclusion

COVID-19 represent a big problem for world, it makes several changes in the human body. There are several systematic studies show that this disease has harmful effects on lungs, heart, kidney etc. The infection is transmitted to the virus through contact or proximity. Therefore, it becomes the most attractive side to use Internet of Things (IoT) applications and smart wearable sensors for remote diagnosis. The internet of things can be used to provide health care, or it can be used to examine and detect patients in epidemics and reduce the spread of the virus between people.

In this paper, we have proposed a novel system for remote detecting COVID-19 disease. It aims to help the doctors to detect the Coronavirus remotely in order to reduce direct contact with patients. The proposed system was composed of three smart sensor called: pulse, thermal monitoring, and blood sensors. The main component of the system is explained in details. The pulse sensor is used to estimate the HR. A mathematical model has been used to describe the transmitted light oximeter and reflected oximeter as in pulse sensor. We also improve the detection of the time series signal by ignoring effects of motion and concentrating on the region of interest (ROI) locations. In the blood sensor, since the earlobe does not contain bone or fat, it was chosen to facilitate the work of the sensor and blood components analysis. Moreover, it contains the largest amount of blood in the body (14% of the blood in the whole body), the effect of different frequencies on the electrical properties of blood and its components become more accurate. The different frequency effect on electrical properties is applied to analyze the blood and its components. For that, we can measure WBCs and RBCs using the bioimpedance analysis. The proposed sensor has consists of the two electrodes to send the current to the earlobe and measure the produced voltage. We use the COMSOL model to measure WBCs, RBCs and Platelets volume based on blood electrical properties and frequencies. For that, we modified a mathematical model to describe the impedance of earlobe in different frequencies (i.e., low, medium, and high). After computing the HR, WBCs, RBCs, and Platelets, the proposed the algorithm is used to distinguish that if the person with symptoms of COVID-19 or not. The proposed system was tested using six humans and compared the obtained results to the manual diagnostics (see Table 2, Table 3). We found that it is more stable and promises to detect COVID-19 remotely without human intervention. The efficiency of the proposed algorithm is provided comparing to the manual diagnostic, because it is not easy to get the data from more corona patient examinations.

The results obtained from the proposed system were compared with the results measured by medical devices, and a small difference was found between the two measurements as in Table 3. For example, an error rate of 2.5% was found in measuring red blood cells, which is the highest error rate. The lowest error is 0.045% in case of platelet size measurement. This indicates that the results of the sensor-based system converge with the proposed medical system. Although the results are promising for the risks that can arise from direct contact with patient, many test cases are need for examination in the future to avoid the contact with corona patient.

Funding

This study was funded by the Deanship of Scientific Research, Taif University, KSA [research project number 1-441-57].

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.