VIRFIM: an AI and Internet of Medical Things-driven framework for healthcare using smart sensors.

After affecting the world in unexpected ways, the virus has started mutating which is evident with the insurgence of its new variants. The governments, hospitals, schools, industries, and humans, in general, are looking for a potential solution in the vaccine which will eventually be available, but its timeline for eradicating the virus is yet unknown. Several researchers have encouraged and recommended the use of good practices such as physical healthcare monitoring, immunity boosting, personal hygiene, mental healthcare, and contact tracing for slowing down the spread of the virus. In this article, we propose the use of smart sensors integrated with the Internet of Medical Things to cover the spectrum of good practices in an automated manner. We present hypothetical frameworks for each of the good practice modules and propose the VIrus Resistance Framework using the Internet of Medical Things (VIRFIM) to tie all the individual modules in a unified architecture. Furthermore, we validate the realization of VIRFIM framework with two case studies related to physical activity monitoring and stress detection services. We envision that VIRFIM would be influential in assisting people with the new normal for current and future pandemics as well as instrumental in halting the economic losses, respectively. We also provide potential challenges and their probable solutions in compliance with the proposed VIRFIM.

Introduction

The novel coronavirus (COVID-19) has affected the world on a large scale and has compelled people to significantly alter the course of their lifestyles. At the time of this writing, COVID-19 has been transmitted to more than 181.6 million people along with unprecedented deaths, i.e., around 3.9 million [1]. Multidimensional changes have been observed in people due to COVID-19 situations such as job loss, educational changes, shortage of supplies, interpersonal relationships, loss of loved ones, fear of infection or illness, occupational stress, social isolation, mental health, and financial distress. Similarly, countries are also facing societal, healthcare, and financial challenges in this pandemic. Patients are experiencing a lack of healthcare services due to the shortage of facilities. The government’s ban on gatherings, congregations, and travel restrictions had a drastic impact on both the country’s economy and the financial dynamics of the citizens. Work from the home strategy was chosen as an alternative which made specific occupations archaic, hence affecting the lifestyle to great lengths.

The contraction of COVID-19 causes loss of smell and taste, breathing problems, cough, and fever, respectively. The extended length of the aforementioned symptoms in vulnerable patients might result in heart issues, respiratory problems, hypertension, organ failure, and in the worst case death [2]. In the beginning, older populations were considered to be at risk, but as per WHO guidelines “the children and young people are not invulnerable to this virus” [3]. Recently, a COVID-19 variant and mutation, i.e., VUI-202012/01, is under investigation by the Genomics UK consortium [4]. One thing which is of great concern is that N501Y mutation has been discovered which allows the virus to get bonded with the human ACE2 receptor using spike proteins. This bondage allows the virus to be more infectious and easily spreadable [5]. However, it is still in question whether it is deadlier than its former version and the existing vaccine (ChAd0 × 1 nCoV-19 and BNT162b2) can be effective for the mutated virus or not. At the time of this writing, there are a total of four variants of concerns flagged by WHO. The latest one is the delta variant that was first found in India, and since then, the delta variant has been the dominant strain in UK as well as USA [6]. Similar to the VUI-202012/01, the delta variant transmits faster and attaches itself to the ACE-2 receptors. It is also assumed that the delta variant can escape the body’s natural immunity in an efficient manner, thus doing more damage than its predecessor variants. Currently, the WHO has also flagged seven variants of interests that might have different symptoms and reactions in comparison with the variants of concerns [7, 8]. According to the study [4, 5], the Genomics UK consortium has revealed that around 4000 mutations have already been recorded in the spike protein and that more mutations will occur as it is naturally part of the evolution process. It has been observed with the seasonal flu vaccines that they need to be altered every year due to the constant mutations; therefore, it is not wrong to say that the vaccines for COVID-19 and future pandemics need respective adjustments with respect to the mutation which prolongs the pandemic itself [5, 9]. Furthermore, it should be noted that the flu vaccine after alteration does not take much time as the base process is available to the manufacturers, whereas for coronavirus the manufacturers are not yet licensed and the base process has not been laid out [10]. Hence, the vaccine is a potential solution, but it might take months or years to eliminate COVID-19 provided that either the vaccine effectively handles the variants or the virus stops mutating altogether.

The experts have been repeatedly asked the question regarding the possible timeline for COVID-19 expulsion. Currently, there are a total of 5 vaccines that have entered phase 3 trails BBIBP-CorV (Sinopharm), Sinovac, Sputnik V from Gamaleya Research Institute, ChAd0 × 1 nCoV-19 from AstraZeneca and University of Oxford, mRNA-1273 from Moderna, and BNT162b2 from the Moderna as well [9]. Out of the five, the data for only ChAd0 × 1 nCoV-19 [11] and BNT162b2 [12] have been published and only a few are approved by the drug regulatory authority for use in this emergency. Some countries in the Middle East such as Bahrain and the United Arab Emirates have approved the usage of BBIBP-CorV, and Russia has allowed Sputnik V to be used widely while ignoring the consequences and safety protocols [13]. Despite the availability of data, the answers to unknown variables remain at large, for instance, the duration of immunity, viral transmission, the safety protocols for a vulnerable group such as pregnant women and elders, and the risks associated with the vaccine’s adverse reactions [9]. The availability of the data regarding these questions is limited to none. Still, the question regarding the logistics, financial, and social implications are not touched upon. According to Anthony Fauci (Head of the National Institute of Allergy and Infectious Diseases), the only prominent and long-term solution to this pandemic is to follow the public health guidelines which include frequent hand wash, indoor activities, physical and social distancing, wearing masks, and enhancing immunity through natural means [14]. It has also been concluded by a recent study [5, 9] that even though the vaccines will play a vital role in controlling the pandemic, the immediate effects will not be observed at all due to the challenges regarding global distribution and lack of data availability. Furthermore, Jonathan Samet (MD dean of Colorado school of public health) in a recent interview suggested that even though an individual gets vaccine, that individual will not be allowed to physically or socially interact as well as the condition of wearing masks still holds; therefore, hygiene, distancing, and masking need to be prioritized [15]. As per the Colorado department of public health and environment, it is still possible for an individual to get mild infection even after vaccination and that they might still be the transmitters of virus [10].

It is evident from the facts provided that the long-term setting for dealing with this pandemic is to slow down the transmission by changing our lifestyle and complying with the public health guidelines. Unfortunately, it has been observed that the guidelines are not being followed by the general population. Furthermore, a global survey conducted from 19 countries recently concluded that 71.5% of people have shown acceptance for the COVID-19 vaccine provided that their employer or the government recommends it [16]. The study also states that the acceptance ratio is higher, i.e., 90%, where the people have trust in their respective governments such as (China, South Korea, and Singapore), but the acceptance drops to 55% for the countries like Russia. One more thing to consider is that the people who showed acceptance rely on the recommendations of the employer or the country which has not been taken into account for the acceptance model. Considering the constraints of logistics, mutations, and vaccine acceptance, personal and public monitoring systems are in dire need which not only helps to provide recommendations regarding the safety guidelines and hygiene, but also helps to improve the lifestyle and immune system at the same time. One way to accomplish the aforementioned task is to perform continuous monitoring while collecting huge amounts of data. With the advent of microelectromechanical systems (MEMS), the sensors have evolved to be small in size as well as effective. The miniaturization of the sensors has allowed the production of wearable sensors that can collect large amounts of data as well as help in improving an individual’s health. Furthermore, combining the use of wearable and miniaturized sensors with ICT technologies such as 5G/6G communication, big data, artificial intelligence, and the Internet of Medical Things will be able to improve healthcare services, personal hygiene, immunity boost, mental healthcare, and contact tracing problems, accordingly. It has been proved by many studies that wearable devices and mobile sensors are widely accepted by the audience varying from children to the elders [17-20]. The integrated services will help to alleviate the COVID-19 and future pandemic related issues on a personal level which will eventually lead to facilitate the financial issues of a country on a macrolevel.

This article focuses on the use of the Internet of Medical Things (Wearable sensor analytics) to deal with physical healthcare monitoring, personal hygiene and immunity boosting, mental healthcare, and contact tracing which are considered to be good practice adaptations to slow down the transmission of infection. We provide possible implementation strategies, the challenges, and potential solutions for each of the aforementioned issues. We provide two case studies related to the physical healthcare monitoring and mental healthcare to show the effectiveness of the proposed framework. To the best of our knowledge, a framework in compliance with the Internet of Medical Things guidelines for long-term setting to deal with current and future pandemics has not been provided. Particularly, the contributions of this study are given below:

Realistic process flows or frameworks for societal issues based on the wearable sensor characteristics.

Analyzing the limitations and challenges for each of the societal issues from the perspective of wearable sensor analytics.

We propose the VIrus Resistance Framework using the Internet of Medical Things (VIRFIM) to help align the good practices in this new normal.

We show the effectiveness of VIRFIM on benchmark datasets for physical healthcare monitoring and mental healthcare, accordingly.

The rest of the paper is structured as follows: We present preliminaries for Internet of Medical Things-based architectures and the layers involved in Sect. 2. We propose hypothetical frameworks for physical healthcare monitoring, personal hygiene and immunity boosting, mental healthcare, and contact tracing in Sects. 3, 4, 5, and 6, respectively. Section 7 presents the details regarding the proposed VIRFIM. Section 8 and 9 present case studies related to the VIRFIM framework. Section 10 highlights some potential challenges and issues when adapting VIRFIM architecture, and Sect. 10 concludes this study.

Preliminary understanding

The following subsections will extensively use some terminologies which are specific to Internet of Medical Things (IoMT) and Internet of Everything (IoE). The IoMT and IoE are derived from Internet of Things (IoT) which combine the network and the medical things or things in general. However, IoE extends the relationship of network and things with people, data, and processes. Some studies referred IoE as Internet of Things, Services, and People (IoTSP) [21]. The core idea and architecture used behind IoE and IoTSP is the same. As we present realistic frameworks for the prevention of COVID-19 and future pandemics, we define some terminologies related to IoMT and IoE, accordingly. It should also be noticed that the conventional IoT architecture comprises of 4 layers; however, this study mainly focuses on IoTSP architecture which uses five layered architecture [21]. The description of each of the layers in IoTSP architecture is given below.

Sensor Layer This layer either acts as a standalone or a collection of memory-constrained, small, and battery operated sensing devices. The scope of IoMT and IoE expands the scope of this layer and considers it as a collection of devices that is connected via Internet and is capable of sensing data.

Access/Communication Layer As the name suggests, this layer comprises of set of communication protocols that can relay the data from sensor devices to the middleware or directly to the server. The communication protocols include but are not limited to sub-GHz proprietary, long range (LoRa) WAN, Zigbee, Bluetooth, radio frequency (RF), long-term evolution (LTE), LTE-advanced (LTE-A), and Wireless Fidelity (WiFi) Direct.

Middleware This layer is mostly considered as a software, but in this study we consider the smartphone as the middleware which acts as an intermediary for collecting the data from sensor layer and sending it to the server layer. Furthermore, the middleware can be used for data management, API management, authentication, messaging, and application services, respectively. In the present architectures, middleware is also responsible for fetching the decisions related to specific services from server layer and send it to the notification center.

Server Layer Although the server layer varies with respect to the evolution of IoT studies, the common aspect of this layer is to receive the data from middleware, process, analyze, and store it to provide specific services. In this study, server layer comprises of two components, i.e., context (bag of contexts) and services (pool of services). The context can either be recognized automatically or derived from statistical inferences (implicit service). The context selection needs to be performed in order to select explicit services (pool of services), accordingly. The service layer is also equipped with decision-making and data analytical techniques.

Notification Center Once the middleware acquires decision based on the explicit services, it is pass to the notification center in form of alerts and recommendations. The notification center also allows the user to visualize the measurements obtained from sensing devices in real time.

The flow of the architectural layers presented in the subsequent sections is given below:

The data are acquired from the sensor layer and sent to middleware via access/communication layer.

The middleware stores the data temporarily acquired from the sensor layer via access/communication layer.

The temporarily stored data are accessed by the context (a component in the service layer) to determine the context. (The determination of context is an implicit service which will be carried out without the user preference or intervention.)

Based on the selected context, an individual service or pool of services will be activated and perform the desired task.

Middleware acquires the decision obtained from the activation of desired service(s).

Middleware sends the result of the activated service to the notification center.

Physical healthcare monitoring

Monitoring of physical human health is of utmost importance during this pandemic, especially if an individual is self-isolated in case of getting infected or due to voluntary isolation after getting vaccinated. It is also a way to offload the bottleneck from the hospitals or healthcare provision institutes that are overwhelmed with the inflow of patients even with mild symptoms. We present a summary of the embedded sensors with smart devices (not limited to) in Table 1. The increase in embedded sensors leads to an increase in price as well. However, some good and not so expensive smartwatches can be used for basic physical healthcare monitoring. Furthermore, some of the digital instruments which monitor the physical health of a patient in an individual manner can also be used for monitoring purpose. There are many aspects of physical healthcare that can be monitored using a smartwatch in connection with smartphones such as physical activity, skin temperature, oxygen saturation, and anomaly detection. Physical activity is an important aspect that not only can be used to check the patient’s activities, but also helps in boosting immunity. We will shed light on the immunity boosting using physical health in the subsequent section. In this section, we focus on the monitoring of physical activities while in isolation and its compliance with the routine suggested by the caregiver/doctor.

Table 1

List of some smartwatches embedded with wearable sensors

The inertial measurement units such as accelerometer, gyroscope, magnetometer, and orientation sensors embedded in smartphones, as well as smartwatches, have been used extensively for physical activity recognition over the years [21-24]. The problem with the adaptation of activity monitoring approaches is the lack of personalized data and the variation in wearable sensor devices in terms of sampling rate, recording units, and so forth. Some studies solve the problem to an extent by either using a semi-population calibration approach so that personalized activity recognition could be carried out with a minimum level of annotated data, or transfer learning approaches that take into account the domain adaptation of varying sensor characteristics. The study [24] proposed the use of behavior for personalized activity recognition using a semi-population calibration approach which not only considers the diverse nature of human behavior when performing certain activities, but also the varying class labels. Recently, the studies [25, 26] have highlighted the problems concerning physical healthcare associated with the current pandemic situation. The study recommended the use of innovative strategies and coordinated efforts from policymakers and health authorities to improve the healthcare systems. Physical activity recognition will allow the monitoring of patients in an automated way without physical interaction with the caregiver/doctor.

Skin temperature and oxygen saturation monitoring are of utmost importance with regard to the COVID-19 pandemic. As COVID-19 (SARS-nCOV-19) is a respiratory syndrome, it causes high fever as well as breathing issues to the infected individual. The fever and respiration issues are the most common symptoms of COVID-19 virus contraction listed by the World Health Organization. Considering that the skin temperature and the oxygen saturation (SpO2) can be monitored continuously through the smartwatches is a relief to both the patients and doctors. In the isolation phase, the spike in skin temperature or decline in oxygen levels might recommend the patient to take necessary action or notify the concerned doctor for immediate response in time. Furthermore, the sensor modalities can be used to log the temperature and oxygen responses in case of voluntary isolation after the vaccination.

The field of anomaly detection using wearable sensors has been in discussion for quite some time. In many countries, elders prefer to live alone which make them vulnerable to some anomalous activities such as fall or slip and also allow them to skip their medications due to memory loss. The anomaly detection in the isolation phase can be used to monitor the same anomalous activities which might alert the authorities in case of sudden fall that require immediate attention or recommend an individual about their medication which needs to be taken at a specified time. There are many studies which propose the use of smartwatch and smartphones for detecting falls accurately [21, 23]. Furthermore, the isolation center or the place where individuals get isolated can be equipped with an object, infrared, or location sensors, to detect anomalous behaviors as well such as sleeping, eating, and physical behaviors that may help the caregivers to understand more about the progression of the virus, accordingly.

We propose a hypothetical framework, physical healthcare monitoring module (PHM) as shown in Fig. 1. The framework is compliant with the sensor devices shown in Table 1 except the beacons which could be used as location sensors. The smartphone acts as a middleware that will be responsible for sending the data acquired from sensors to the service layer for specific service provision. The smartphone is also responsible for fetching the decision obtained from the service layer and displays it to the mobile application interface or send the decision to the doctor/caregiver for necessary action. The steps indicated by the small letters in Fig. 1 are compliant with the steps mentioned in Sect. 2. The PHM is one of the modules which could help in monitoring a patients’ health while in the isolation phase which is an essential phase if one gets infected or vaccinated alike. Furthermore, the module can transform even an individual room of a distant house into an isolation ward in terms of healthcare service provision.

Fig. 1

The physical healthcare monitoring module for patient monitoring during the isolation phase. ST Skin temperature, SpO2 oxygen saturation, PAM physical activity monitoring, AD anomaly detection

Personal hygiene and immunity boosting

According to the World Health Organization's public safety guidelines with regard to the COVID-19 pandemic, personal hygiene is of utmost importance in order to slow down the spread of viral infection. Personal hygiene includes washing hands with water and soap for more than 20 s, use of sanitizer in case the former option is not available, repeating the steps if cough or sneeze is detected, accordingly. Furthermore, it is not just sneezing or coughing that releases the droplets, even talking to someone or having a conversation can release the droplets as well. Therefore, having a recommender system that reminds an individual for taking care of personal hygiene can play a vital role in reducing viral transmission. When it comes to wearable sensors, personal hygiene can be monitored and recommendations can be suggested using a camera from smartphones, a microphone from smartphones as well as smartwatches and the inertial measurement units. The IMUs in smartwatches will help in detecting the washing hands and brushing teeth actions, and the recommendations might be provided at specific timelines to repeat the respective actions.

The smartphone camera could be used to scan the barcode of the sanitizer in case the option of washing hands is not feasible. The barcode will provide the information to the users about the alcohol level and recommend its usage as per the WHO guidelines. The crucial part of such a recommender system related to personal hygiene will be based on microphones available in smartphones and smartwatches. The acquisition of sound waves can distinguish between the cough, sneeze, and normal sounds. In case of cough and sneeze sound detection, the user can be provided with the recommendation of washing hands again or using sanitizer immediately. Furthermore, as per WHO recommendation, the distance of the droplets from cough or sneeze depends on several factors such as a person having a full set of teeth, blocked nose, larger viral load, and louder voice. These factors determine an individual as a super-spreader as their sneeze can travel up to 60% further and can produce 4 times more droplets [27, 28]. If such information can be gathered through a profiling system, the recommendation based on user profiles and the microphone sensor can be made regarding the physical distance that needs to be maintained. Another aspect of maintaining personal hygiene is proper ventilation and maintaining good air quality at the place of isolation. Many mobile sensors can be directly paired with smartphones to provide information regarding air quality so that a feasible recommendation can be provided.

A crucial aspect of being probably safe during this pandemic is to boost the immune system through healthy foods and diet. Students have proved that the use of water in large quantities, the use of zinc and magnesium, foods rich in vitamins C, D, and E, herbs, and the use of some specific ingredients can improve an individual’s immune system. However, it is not always possible for everyone to consume such enlisted food items for immunity boosting due to allergies, health constraints, and so forth. Most of the food items in the marts have a barcode or QR code that provides the information related to the ingredients used in the product. The camera in a smartphone can easily capture the bar or QR code to scan the items used in the product and provide a recommendation of its usage in compliance with the health standards and the individual’s profile so that the respective allergies and health issues can be kept in check. Several smartphone apps provide such information while scanning the aforementioned codes such as myfitnesspal1 and more. A question can be made that homemade foods do not come with any bar codes or QR codes. In that case, the natural language processing characteristics can be used to search the ingredients in a certain food item online in an automated way as well as providing a voice command to search for an immunity boosting recipe, accordingly. The use of microphones and the keyboard input either from a smartphone or smartwatch can be leveraged for the said purpose. Google’s speech recognition and search packages in python help in performing such tasks.2 As mentioned earlier, physical activities can also help in boosting immunity. According to studies [29-31], physical exercise can help flush bacteria out of airways and lungs which reduces the chances of catching flu, cold, or other bacterial infections. Physical exercises can also help in changing white blood cells and antibodies which are an essential aspect in dealing with viral diseases. Similar to the food items, all physical exercises are not meant to be performed for individuals due to their body types, health constraints, and other issues. Based on an individual’s profile, the system can recommend the type of exercises that are considered to be safe.

The hypothetical framework for personal hygiene and immunity boosting (PHIB) module is shown in Fig. 2. The framework only opts for the wearable and mobile sensors that could be connected via smartphone for data acquisition. The framework is also compliant with the Internet of Things (IoT) layers for real-life applicability. The data from multiple sensors are acquired and stored in a temporary database in the middleware. The camera can help in reading bar and QR codes, respectively. The microphone sensor can be used to detect cough or sneeze sounds as well as recording users’ queries for relevant exercises and food ingredients. The GPS sensor in the smartphone helps recognize the context as the person is indoor or outdoor. The indoor location context will activate a certain set of services such as recommendations for indoor exercises, cooking, hand wash by detecting coughing and sneezing sound at home, air quality index, and more, while the outdoor location triggers physical distance alert, foods available in restaurants, sanitization alert, and outdoor exercises. The data are then pushed to the server for further processing, detection, recognition, and recommendation, accordingly. The user gets notified on their mobile screens for the desired service.

Fig. 2

Personal hygiene and immunity boosting module for indoor and outdoor locations

Mental healthcare

A surge in stress, anxiety, and depression has been observed as an after effect of the COVID-19 pandemic. Although the pandemic is not over, the stress and anxiety continue to play a causal role in mental health conditions. Researchers have even related the emotional and behavioral response of individuals in this pandemic to the terror management theory where the fear of death plays an important part in making daily life decisions [32-34]. Furthermore, many studies have concluded that self-isolation may lead to prolonged stress which might result in anxiety, distress, or depression, depending on the length of the isolation period. Studies have also suggested that the fear and anxiety in the times of COVID-19 have increased the crisis on public healthcare services. Recently, Centers for Disease Control and Prevention (CDC) has also emphasized managing stress and anxiety levels as stress during the infectious period can result in abnormal eating and sleep patterns, an increase in chronic health issues, reduced immunity, and panic attacks [26, 35, 36]. According to CDC, elders, frontline health workers, socially isolated, people who lost their loved ones, people who lost jobs, and people facing financial crisis might react strongly to stress crisis during the pandemic. Another recent study [37] suggested that as the stress gets prolonged, it could result in suicidal behavior if not intervened with a timely and relevant response. Reports from CDC, John Hopkins University, and Boston University of public health have concluded that anxiety and depression have been tripled and quadrupled this year in comparison with the preceding years [38, 39].

According to CDC, coping with stress will not only be helpful for an individual’s health, but it would affect the overall community in terms of extended empathy, voluntary support, increased social connection, and less strain on public health services. However, in achieving community-wide benefit the emotional distress needs to be diagnosed in time. Mental health disorders such as emotional distress and stress can be measured with wearable sensors including electrocardiograph (ECG), GSR, and electroencephalogram (EEG) [21, 40, 41]. Some of the smartwatches listed in Table 1 comprise of such sensors that could help in recognizing the emotions of an individual, accurately. However, mobile sensors for such modalities are also available commercially which could be connected with smartphones for continuous data collection. Timely stress recognition may notify doctors/caregivers, family members, friends, loved ones, and volunteers to help relieve the stress of an individual. Moreover, timely detected stress can also be handled with home automation as COVID-19 is not the only cause for inducing anxiety. Some studies have also considered the usage of smartphones and posts on social media to determine stress levels. The use of natural language processing can be leveraged for determining stress as well while considering text or voice conversation with the chatbot as an input.

Figure 3 presents a hypothetical framework for the mental healthcare (MHC) module. The middleware can acquire the data from the wearable and mobile sensors on a continuous basis and store the data temporarily in the buffer for some seconds before sending it to the server. The server will be able to detect anomalous emotions such as anxiety, depression, distress (prolonged stress), and suicidal tendencies. The context of isolation and non-isolation is selected based on the studies [32, 34, 38, 39] which concluded that the stress related to self-isolation, infection in COVID-19 pandemic has a different characteristic than that of financial, job, and occupational stress. The context of pre- and post-COVID infection can also be taken into account for the MHC module. Based on the detected anomalous emotion, multiple actions/recommendations can be provided such as variations in lighting conditions, calling to family or friends, having a conversation with an emotional chatbot, and so forth. Furthermore, this system can also be employed for stress in people driving cars or with long commute times for suggesting them an alternative fast route.

Fig. 3

Mental healthcare module for isolation and non-isolation phases

Contact tracing

The importance of contact tracing was highlighted when “patient 31” in South Korea infected hundreds of patients over the span of days. With the outbreak of COVID-19 manual contact, tracing procedures were performed by the health departments of different countries. The process is time extensive, prone to errors, and is not scalable and, thus, appears to be ineffective so far. Since then, government officials and policymakers have moved toward a digital solution based on intelligent decision support systems. Recent studies [42-44] have urged the use of smartphones and mobile sensors for contact tracing in order to limit the transmission of the virus, hence resulting in smart lockdowns. Although solutions have been provided for proximity tracing and alerts, the rollout plan and tracing of contacts have been limited due to mainly two reasons. The first is the tracing only using GPS sensors using smartphones, and the second is the lack of behavioral consideration. The first-ever contact tracing protocol for dealing with the COVID-19 pandemic was developed by Singapore’s government which used Bluetooth to perform the respective tracing. The protocol was based on OpenTrace,3 an open-source implementation of Android and iOS apps along with baseline calibration data and cloud server backend. The protocol can be called by its alternative terms such as BlueTrace and TraceTogether. The Australian government in April 2020 launched an app that traces the contact manually that complies with the BlueTrace protocol and was termed as COVIDSafe.4 The Chinese app (Chinese health code system), South Korean app (Corona 100 m), and the UK’s (NHS COVID-19) app have been proposed with similar characteristics. Research studies with different ways of contact tracing such as leveraging healthcare reports, clinical documents, and barcodes for location similarity have also been proposed [45, 46]. There have been other studies that proposed the contact tracing algorithms, but face the same fate of being ineffective when it comes to scalability. A recent study [24] highlighted the importance of behavior analysis when it comes to activities of daily life. The use of behavior analysis can be incorporated with the activity recognition using wearable sensors to derive context rather than just tracing the persons gathered at a common place. A person who works as a waiter in a restaurant performs an activity and exhibits behavior differently than the one who visited for having lunch or dinner. Both of them have different levels of exposure while coming into contact with an infected individual. Furthermore, the use of natural language processing can be garnered in order to get additional information about the places where an individual has traveled. Furthermore, the ones who visited the probable infected places can be notified in the same time span. The activity combined with the given context can reveal whether the person has just passed by a certain restaurant in a vehicle or by walk which again provides a different perspective of infection spread via an individual.

With the emergence of new COVID-19 variants and mutations, the importance of contact tracing has been raised manifold. It is a necessity to keep the track of individuals that have traveled back and forth from a certain location where the variants are prominently discovered. Moreover, such a kind of contact tracing will help to identification of vaccine variants that need to be injected with respect to the location of the mutation. The GPS sensor along with inertial measurement units, speech analysis, and social media content, the automation of contact tracing could be improved with real-life applicability.

We present a hypothetical framework for the contact tracing (CTC) module in Fig. 4. The CTC block considers the data from GPS sensors, inertial measurement units, voice input, and travel history of individuals using the system. The data are temporarily stored in the middleware and is pushed to the server for necessary detection based on the location, time, and travel history contexts. The location and time context have been used to integrate behavioral characteristics when recognizing high-level activities; therefore, the recognition of activities at specific locations can provide insights of a person either infected or not. With the unified location history of the individual using the system, contact tracing can be made easy based on the intersection of locations. Furthermore, in the case of a person traveling from a specific area where a new mutation or variant of COVID-19 has been found, an alert can be generated to users in the travelers' proximity. The system can also highlight locations where the COVID-19 infection risk is high so that the users may avoid the route or place altogether. An additional feature of distance measuring can be added between the smartwatches of different people to recognize handshake activity which could help trace the users in case of either one being infected in prior.

Fig. 4

Contact tracing module based on the location, time, and travel history

VIrus Resistance Framework using the Internet of Medical Things (VIRFIM)

Based on the four hypothetical modules, we propose a general hypothetical VIrus Resistance Framework using the Internet of Medical Things (VIRFIM). It can be noticed from the modules that more or less the sensor, access/communication, and middleware layer are similar. However, the context and server layer vary with respect to the targeted module. Furthermore, a general dashboard in an app could be constructed for either automatic or manual activation of a particular service. The proposed COFIE framework is shown in Fig. 5. We briefly define each of the layers in the VIRFIM framework.

Fig. 5

Proposed VIrus Resistance Framework using the Internet of Medical Things (VIRFIM)

The sensor layer in the VIRFIM framework is responsible for data acquisition from various wearable and mobile sensors. The only constraint we put on the selection of the sensors is their ubiquity and pervasiveness. Although a single sensor device may house multiple sensors, for instance, an inertial measurement unit might comprise of accelerometer, gyroscope, magnetometer, and other sensors as well, the sensor layer of VIRFIM framework considers each of the sensor measurements separately despite the device used or embedded in. The reason for considering individual sensor measurements is due to the diversified characteristics of embedded sensors and associated applications. For example, the breathing sensor can be used for anomaly detection in an isolation phase, whereas the same sensor is used for physical activity or exercise recognition in the non-isolation phase. In this regard, the VIRFIM framework assumes that there are sensor devices, i.e. and sensor types in each , i.e. . Using the sensor device and its types we can define an individual sensor measurement as shown in Eq. 1.where refers to the individual measurement of sensors, is the unique id for each individual measurement from sensors, and are the sensor device and sensor type, respectively. The acquired IMS will then be transferred to the middleware via the access/communication layer. This layer acts as a source of communication between the sensor layer and middleware by using abstract protocols such as serial peripheral interface (SPI), inter-integrated circuits (I2C), Zigbee, infrared, and Bluetooth. The VIRFIM is a general framework; therefore, the protocols in the access/communication layer are not limited to the aforementioned ones and can be extended to other protocols such as RFID, depending on the given context and service.

As the VIRFIM framework is mainly designed considering the wearable and mobile sensors, the smartphone is considered to be the middleware that acts as an intermediary device for all the layers, respectively. The data acquired from the sensor devices via the access/communication layer are stored temporarily in the memory which then is pushed to the server via a gateway or 5G/6G services. Once the data are pushed, new data acquired from the sensors will be overwritten, accordingly. It has been proved in the existing studies that the storage of data in memory temporarily is a lightweight operation [21]. The preprocessing of for opting context and triggering desired service will be performed on the server layer. The decision from the desired service is then collected from the middleware while using REST API endpoints to call a specific function. The decision is then either shown to the application dashboard or sent to the doctor/caregiver, respectively.

The bag of contexts is of vital importance and is considered to be an implicit service in the VIRFIM framework. The consideration of context from can be performed either using knowledge-based or data-driven approaches. Some studies also considered the use of hybrid methods which leverages the characteristics of both the knowledge- and data-driven approaches for recognizing the context in an automated manner. The knowledge-driven approaches use resource description framework (RDF), web ontology language (OWL), and simple protocol and RDF query language (SPARQL), whereas the data-driven approaches use machine learning techniques to perform the desired task. It should be noticed that the VIRFIM architecture is not limited to the suggested contexts such as time, location, action, and therefore can be extended to various other contexts depending on the service and availability of sensors. The bag of contexts in the VIRFIM framework is represented as where represents the number of available contexts. Once the individual measurement from sensors is obtained the data could be sent to the pool of services for activation based on the selected context. The pool of services might include a web server for a decision support system, a self-designed app in a smartphone, or a third-party app such as Samsung Health and myfitnesspal. Furthermore, the pool of services will implicitly store the data for summarizing the data, provision of logs, and alerts for the use of medical assistance. The pool of services in the VIRFIM framework is defined in Eq. 2

The and refer to the selection of a specific service and its unique id, respectively. It should be noticed that similar context can be associated with multiple services; for instance, location and time contexts are considered for the services listed in PHMM and CTC modules. Therefore, multiple services can be triggered with respect to the selected contexts. A search algorithm for initializing a particular service is given in Table 2. The algorithm looks into all the services available (Line 2), lists the services with similar context requirements (Line 4), and for all similar services (Line 5) checks the availability of the sensor measurements (Line 6). In the case of available measurements, the VIRFIM will activate the desired service, respectively.

Table 2

Search algorithm for service selection in VIRFIM framework

	Search algorithm
1	Initializing pool of services
2	PoS=pos_n|n=1,⋯,N
3	For pos_n in PoS
4	CSV=pos_z|z=1,⋯,Z, where Z≤N and CSV⊆PoS
5	For pos_z in CSV
6	Check IMS_ε,n
7	IF sensor measurements available
8	Activate the service

In the above example, five individual sensor types from four sensor devices have been acquired. Based on the context two services, i.e., physical activity monitoring () and COVID-19 tracking in the area of your activity (), are selected automatically.

Case study 1 (physical activity recognition)

The previous section highlights the technical details of VIRFIM framework for using context to provide particular services. It should be noted that the VIRFIM framework is not limited to any specific machine learning algorithm and can employ knowledge-, data-driven, or hybrid learning approaches. For this case study, we consider PAMAP2 dataset [47] and hybrid learning approach [21]. The hybrid learning approach suggests that a prior information is available (knowledge-driven) for defining ontologies that would provide the contextual information, followed by a data-driven approach for identifying the high-level activities. The employed dataset comprises of two IMU sensors and one physiological sensor able to record heart rate and skin temperature. The measurements from each of the IMUs was acquired using 100 Hz sampling rate while the heart rate and skin temperature readings were obtained at 9 samples per second. The PAMAP2 dataset has annotations of both the high-level activity and location information and, therefore, is considered to be compliant with the proposed VIRFIM framework, accordingly. We define the ontologies as suggested in [21, 48]. The data-driven approach will follow a conventional machine learning pipeline, i.e., preprocessing, feature extraction, and classification. For this case study, we consider the location information as an implicit context and high-level action recognition as an explicit service. The selection of the context and services are compliant with the studies [21, 24]. The relationship of context with the high-level activity information is shown in Table 3 and an abstract flow in accordance with VIRFIM framework is shown in Fig. 6, respectively.

Table 3

Predefined ontologies for location and high-level activities

Fig. 6

Physical activity recognition for PAMAP2 dataset using VIRFIM

To validate the applicability of VIRFIM for physical activity recognition using wearable sensors, we preprocess the PAMAP2 dataset as follows:

The sensor IDs are removed as the VIRFIM stores the sensor measurement, implicitly.

Although the time can also be considered as an added context, for the sake of this case study we will ignore the time stamp field in PAMAP2 dataset.

To match the sampling rate of IMUs and physiological sensors, we down sample the IMU data to 10 Hz by taking the average from 10 sliding windows and then fill the last available value for physiological sensor data with respect to the corresponding window.

After preprocessing, we consider a total of 52 attributes and one label column.

Once the data are preprocessed, we extract features from both the IMU and physiological sensor measurements. We extract 12 features from each of the IMU measurements while 11 features from the physiological ones as suggested in [21]. The feature extraction includes statistical, structural and transient features, respectively. The data are trained and evaluated for seven classification techniques including decision trees, support vector machines, random forest, extreme learning machines, extreme gradient boosting, 1-D convolutional neural network (CNN), and long short-term memory network (LSTM) [21, 24, 41]. The reason for opting the aforementioned classification methods is their extensive used for cloud and mobile applications in existing studies [49-51]. The evaluation is carried out using leave-one-subject-out validation scheme and classification accuracy, accordingly. We use grid search method to select the optimal hyper parameters for each of the classification methods. The architecture of 1-D CNN is kept simple, i.e., two CNN layers, followed by dropout and pooling layer. Similarly, for LSTM we used a single layer with 256 hidden units. The optimization algorithm for both CNN and LSTM was set to ADAM with the default values. The learning rate, number of epochs, and other optimizable parameters were selected using grid search approach, accordingly. Furthermore, we also used some state-of-the-art deep learning approaches such as DeepSense [52], and DRBLSTM [53], for showing the compliance of VIRFIM architecture with advanced deep learning strategies. It should be noted that we used the default parameters for DeepSense and DRBLSTM when training the network on PAMAP2 dataset. The results are reported in Table 4. The results show that the best accuracy is achieved using DRBLSTM, but with higher inference time in comparison with most of the classification methods, whereas the lowest inference time is achieved using decision trees. The selection of classification method should be based on a trade-off depending on the characteristics of the implemented system. For instance, soft real-time systems are flexible with the deadlines and execution times so 1-D CNN or LSTMs could be opted; however, if the system is designed on the hard real-time characteristics then extreme learning machines would be an appropriate choice. There are several services that could be implemented in line with the physical activity monitoring case study such as fall detection [23] and health anomaly detection [41], but due to the unavailability of publicly available datasets that include contextual information, we have not considered it for analysis.

Table 4

Classification accuracy and average inference time for activity recognition using various classification methods

Classification method	Accuracy (%)	Average inference time
Decision trees (CART)	69.34	0.23 ms
Support vector machines	68.05	0.4 s
Random forests	74.69	0.89 ms
Extreme learning machines	82.47	0.44 ms
Extreme gradient boosting	81.58	0.58 ms
1-D CNN	86.39	0.54 ms
LSTM	88.42	0.82 ms
DeepSense	88.57	1.36 s
DRBLSTM	89.45

Case study 2 (Mental healthcare)

In this work, we perform the second case study related to mental healthcare, specifically stress detection service using VIRFIM. We employ the dataset related to stress detection from the study [41]. Although there is no prior context available, authors do mention that the participants are factory workers and graduate students. In this regard, we assume a prior context of physical and mental work for making the analysis compliant with VIRFIM framework. It should be noted that there are other datasets related to stress recognition, available. However, those datasets are either not available publicly or they are not collected using wearable sensors. The employed stress detection dataset was collected using physiological sensors embedded in a smartwatch which makes it compliant with the performed study. The data were collected from 14 participants (5 females and 9 males) with an age range of 20–38 using heart rate variability (HRV) and galvanic skin response (GSR) sensors embedded in Microsoft Band 2. We use the same hybrid learning approach for stress detection service. We first provide a hypothetical context of work type, i.e., physical and mental (based on the demographic division in the data collection study). We then use the context to detect “physical stress,” “mental stress,” and “no stress” using data-driven approaches, accordingly. The data were acquired using HRV with 1 Hz sampling rate and GSR with 5 Hz sampling rate; therefore, we apply some preprocessing measures which are as follows:

The GSR value range varies with respect to each subject; therefore, min–max normalization was applied to scale the values.

We down sample the GSR values to 1 sample per second in order to match the sampling rate of HRV.

We divide the original label “stress” into two categories, i.e., “physical stress” and “mental stress” based on the subject characteristics.

After preprocessing, we consider 2 attributes and one label column.

As per the literature, mostly time-domain, frequency-domain, and specific modality features are extracted from the physiological sensor data [40, 41, 54]. An example of specific modality feature for HRV and GSR includes RR interval and signal power of the skin conductance, respectively. We used the same features as suggested in the original study. The data are trained and evaluated using the same classification algorithms as mentioned in previous section. The evaluation is performed with classification accuracy while following the leave-one-subject-out protocol. The classification algorithm parameters are optimized using grid search approach. We report the results in Table 5, accordingly.

Table 5

Classification accuracy and average inference time for stress detection using various classification methods

Classification method	Accuracy (%)	Average inference time
Decision trees (CART)	77.84	> 0.1 ms
Support vector machines	78.98	0.9 ms
Random forests	79.56	0.43 ms
Extreme learning machines	84.66	0.13 ms
Extreme gradient boosting	78.75	0.27 ms
1-D CNN	92.57	0.29 ms
LSTM	88.36	0.54 ms
DeepSense	92.89	1.44 s
DRBLSTM	93.66	1.13 s

Based on the obtained results and real-time system characteristics, we conclude that the 1-D CNNs qualify for best trade-off strategy as it achieves highest accuracy while having inference time lower than the DRBLSTM. However, if the hard real-time system is to be considered then extreme learning machines would be a better choice. An abstract flow of stress detection service in accordance with VIRFIM framework is shown in Fig. 7.

Fig. 7

Stress recognition for the dataset collected in [41] using VIRFIM

Issues and challenges

Despite the use of IoE and IoMT, a number of challenges including privacy, security, scalability, and quality of service might impact the use of the proposed solution. We briefly highlight the issues and their probable solution.

Challenge related to privacy Data privacy is of major concern for the VIRFIM framework as the data gathered from wearable and mobile sensors contain personal information and an individual would not risk the data privacy even if it helps to protect him/her from the contagious virus. Furthermore, information theft due to the privacy breach can be used for illegal benefits. The privacy issue has been highlighted by some legal frameworks such as the health insurance portability and accountability act (HIPAA) [55] and general data protection regulation (GDPR) [56].

Potential solutions and research directions for privacy concerns To deal with privacy issues, one of the solutions is to move the analytical processes to the middleware or edge devices having lightweight operations for critical or highly sensitive data while transmitting only the decision label to the server for further action. Another solution to the privacy issue is the use of software defined privacy [57], privacy by design [58], Federated learning [59], and other solutions in compliance with the privacy life cycle.

Challenge related to security The security issues have garnered a lot of attention from researchers recently due to the versatility of attacks and fast pace modifications. The data collected from wearable sensors are quite sensitive due to the high-risk involved with the decision analysis part. Such attacks could manipulate the data for the change of decision at the analysis stage which might result in actions involving risks or negative impact. The security issues are also directly related to scalability suggesting that the security gets vulnerable with the increase of devices.

Potential solutions and research directions for security concerns One of the possible solutions is similar to the privacy concern, i.e., to move essential services to the edge devices or middleware for reducing the communication flow of the data to the server. The VIRFIM framework can be fused with a secure REST approach or lightweight anonymous authentication protocol to increase network security. Conventional authentication mechanisms such as IDs and passwords can be used for securing the data. However, for more efficient approaches the use of machine learning and distributed services such as Blockchain can be leveraged to secure the data, accordingly.

Challenge related to scalability and quality of services As the number of wearable sensors and devices is increasing drastically, it poses scalability and quality of service issues to a great extent. Similar to the devices and with the emergence of IoMT and IoE, the number of users and other service requesting entities have also increased manifold. The service provision to all the users and devices lead to network congestion problem which could probably delay the decision outcome of VIRFIM framework. In this regard, the framework and the network service provider both need to support scalability in order to facilitate hard and soft real-time systems, especially in these desperate times. With reference to the increase in devices, it requires the framework to handle the data generation and storage process.

Potential solutions and research directions for scalability and quality of service concerns Distributed data storage services could be used which include Cassandra, MongoDB, and Apache HBase. Keeping in mind the VIRFIM framework, the middleware could use “s3cmd utility” to send the acquired data to Amazon storage in a distributed manner which then will be permanently stored to the online server. For network scalability, many methods based on machine learning and optimization have been proposed for active user detection in 5G/6G networks. This benefits the VIRFIM framework in two ways. The first is the reduction of communication between devices that are not involved in the desired service being selected and the second is the grant-free access to the base station in an efficient way. Moreover, virtual software network functions and network slicing approaches can also be used to reduce the network congestion problem, respectively.

Challenge related to society Societal issues due to the continuously revolving conspiracy theories and technology acceptance have been there for most of the advancements that have happened in the last decades. Recently, the link of COVID-19 and 5G technology has made a lot of headlines which not only impacts the technology acceptance, but also affects the intrinsic issues of a particular new technology and advancements in the specific directions.

Potential solutions and research directions for societal issues As mentioned earlier regarding the acceptance of the COVID-19 vaccination, the employers and the governments could be taken in confidence in order to provide technical literacy regarding the breakthrough or advancements. The collaboration with governments and employers could reduce the impact of conspiracies and help the people to accept the technology which is mainly proposed to assist people with their everyday activities as well as facilitate them in protecting from the contagious virus.

Challenges related to legal implications Whenever the collection of data is involved, especially if the same data are used to track your activities without your consent, the legal issues will pave their way. Furthermore, the collection of such data without consent have legal implications. This problem is also related to technology acceptance and so does its solution.

Potential solutions and research directions for legal implications Similar to the participants who are willing to volunteer for vaccines, employers, and government organizations, in collaboration, can call for participants who are willing to volunteer for providing their data regarding contact tracing. This procedure could also involve standardization and regulatory bodies to provide guidelines regarding the collection and use of such data. It will not only reduce the legal implications, but will help individuals (who have faith in their employers and governments) to understand the importance of such systems.

Conclusion

The future is unpredictable but so do the events that lead to that future. It has been established that vaccines for coronavirus and in case of future pandemics will either take time to reach out in every corner of the world or need continuous modifications/alterations due to the mutations and variants. In the latter case, the life cycle of vaccine developments needs to be revised along with the time to get approval from the drug regulatory authority. It has also been learned in previous months while dealing with pandemics that if one occurs in the future, the solution of vaccines will take at least 2–3 years for reaching to an individual (depending on the several medical and technical constraints) and that the good practices, immunity boosting, personal hygiene, healthcare monitoring, and contact tracing might help in slowing down the transmission of infection. The aforementioned good practices can be brought to realization with macrosolutions based on wearable/mobile sensors and integrated technologies.

In this article, we emphasize the importance of using integrated technologies to help in dealing with the COVID-19 and future pandemics till the vaccine shows up at the doorstep. We presented hypothetical frameworks for physical healthcare monitoring, personal hygiene and immunity boosting, mental healthcare, and contract tracing applications. Furthermore, we presented VIrus Resistance Framework using the Internet of Medical Things (VIRFIM) which combines the aforementioned modules into a single framework. We briefly defined the technical details and provided a summary of potential challenges along with the probable solutions and research directions to make VIRFIM a realization. We assume that VIRFIM in general can prevent individuals from contracting novel coronavirus while adopting good practices. We believe that VIRFIM will not only highlight the importance of using wearable/mobile sensors for individual prevention, but also will help in making statuary bodies, governments, and industries redirect some part of their funding to the proposed initiative.

We further validate the use of VIRFIM framework for physical activity monitoring and stress detection services with the help of case studies, respectively. It is shown that by using the contextual information, the performance of employed service could be enhanced. It was further discussed which of the machine learning algorithms could be best suited based on the real-time system characteristics and classification accuracy, accordingly.

One of the limitations of this work is that there are several other constraints, challenges, and research directions that include connectivity issues, battery consumption of the middleware and wearable sensors, memory profiling, and others which have not been touched upon in this work. We believe that the proposed work will also shed light on potential future works that could be branched off from the VIRFIM architecture. We intend to develop an Android app based on the VIRFIM architectural attributes and to show the potential benefits while avoiding any legal implications in terms of data collection and analysis. We also intend to conduct an agent-based simulation that could help understand the economic benefits if the VIRFIM architecture is used at a large scale.