A lightweight federated learning based privacy preserving B5G pandemic response network using unmanned aerial vehicles: A proof-of-concept.

The concept of an intelligent pandemic response network is gaining momentum during the current novel coronavirus disease (COVID-19) era. A heterogeneous communication architecture is essential to facilitate collaborative and intelligent medical analytics in the fifth generation and beyond (B5G) networks to intelligently learn and disseminate pandemic-related information and diagnostic results. However, such a technique raises privacy issues pertaining to the health data of the patients. In this paper, we envision a privacy-preserving pandemic response network using a proof-of-concept, aerial-terrestrial network system serving mobile user entities/equipment (UEs). By leveraging the unmanned aerial vehicles (UAVs), a lightweight federated learning model is proposed to collaboratively yet privately learn medical (e.g., COVID-19) symptoms with high accuracy using the data collected by individual UEs using ambient sensors and wearable devices. An asynchronous weight updating technique is introduced in federated learning to avoid redundant learning and save precious networking as well as computing resources of the UAVs/UEs. A use-case where an Artificial Intelligence (AI)-based model is employed for COVID-19 detection from radiograph images is presented to demonstrate the effectiveness of our proposed approach.

Introduction

Recently, due to the devastating spread of the novel coronavirus (COVID-19) on a global scale [1], [2], researchers started exploring how to formulate intelligent pandemic response networks [3], [4]. Communication networks, particularly unmanned aerial vehicle (UAV)-integrated fifth generation and beyond (B5G) mobile networks, can be leveraged to construct agile pandemic response networks to disseminate pandemic recommendations, facilitate digital contact tracing, monitor the spread of the pandemic, and so forth. Incorporating intelligence into the UAVs using Artificial Intelligence (AI) techniques is, however, posing computational and privacy challenges. The surge of pandemic-related traffic will also lead to increasing the communication delay and overwhelm the backhaul links. Therefore, to construct an efficient UAV-based pandemic response network to complement the terrestrial cellular infrastructure for enhanced service coverage, collaborative learning by the UAVs [5] in a lightweight, as well as privacy-preserving manner, is essential.

Even though the drone resources for processing, memory, and battery have improved much recently, the UAVs are anticipated to still lack, in B5G networks, the resources to execute intelligent and coordinated learning based on the monitored and collected pandemic data of mobile users (UEs). Because maintaining privacy is a big concern for the acquired health data of the UEs, existing methods using explicit user reporting on their locations and pandemic data may not be directly adopted. It is worth clarifying the difference between the security and privacy of patient data. In the context of this paper, security typically implies safeguarding the data of the users/patients against unauthorized access of the data. Security measures can be enforced by the state-of-the-art cryptography to limit which stakeholders may have access to the information. On the other hand, the term privacy is harder to define since user-specific details (e.g., user/patient identity and other sensitive data) can also be secure data. In the context of our work, privacy focuses upon masking the user/patient-specific information. Thus, when the patients’ devices, referred to as UEs, communicate with the B5G terrestrial/aerial base stations (gNBs) and provide their raw radiograph images for pandemic data analytics, there are scopes for mapping back the patient data to individual users and reveal their identities, whereabouts, living patterns, and so forth. The cloud-based analytics could also reveal certain information pertaining to the identification of the users [6]. The idea of this paper is to use an asynchronously updating federated learning framework to thwart such threats whereby the wisdom of the patients’ radiograph data are shared rather than the explicit images to train a central AI model without exposing the user-specific details in B5G networks. In this paper, by addressing the privacy challenge, we propose a collaborative learning method exploiting two-layer federated learning [7]. In our proposed technique, the general and specific weight parameters of the AI models locally learned in two layers (UEs and UAVs) are asynchronously updated to update the global AI model at the cloud, managed by hospitals and public health authorities. While reducing B5G network overheads, the proposed technique still allows the cloud to predict the appropriate pandemic features with reasonably high accuracy without the need to receive raw health data, locations, and other information of the UEs.

The key contributions of our work in this paper are outlined below.

To provide a distributed privacy-preserving data collection and analytics framework at the same time, a federated learning-based framework is presented. This facilitates the exchange of wisdom of the underlying B5G mobile user data instead of explicitly sharing the raw data (e.g., location, mobility, health status, and so forth) with the central cloud.

Asynchronous update of weights of local models is considered in the envisioned federated learning framework to significantly improve the communication overheads. This facilitates lightweight decentralized model construction and execution at the resource-constrained mobile UEs and other distributed/local nodes (LNs).

The need to seamlessly merge the asynchronously updating federated learning framework with the blockchain-enabled centralized aggregator is identified as an open research issue that will stimulate much work in the area of privacy-preservation of big medical data.

The remainder of the paper is structured as follows. Section 2 provides the related research endeavors for formulating pandemic response networks. Section 3 contains our considered network model, and then provides a formal problem description. Our proposed two-layer federated learning technique leveraging the asynchronous update of weights is delineated in Section 4. Section 5 presents the performance evaluation of our proposal based on a specific use-case. The future work and open research issues are discussed in Section 6. Section 7 concludes the paper.

Related work

In this section, we survey the recent related work from two perspectives, i.e., the relevant progress in theoretical and applied federated learning in the context 5G/B5G networks, and then review contemporary works on exploiting such federated learning frameworks to combat COVID-19.

First, we aim to capture the recent endeavors on federated learning theories and their applications in 5G and beyond communication networks. The federated learning concepts with privacy issues were discussed in the recent research works in [8], [9], [10], [11], [12]. While the research in [13] surveyed federated learning concepts in 5G/B5G mobile edge networks by considering communication overheads, resource assignment, privacy, and security, other researchers investigated the systems diversity, privacy concerns along with communication costs in federated learning environments [14], [15]. The work conducted in [16] exploited homomorphic encryption, secure multiparty computing, and differential privacy to provide privacy-preservation in federated learning. However, this work and similar surveys in [17], [18] did not address the privacy issues implied in federated learning frameworks. Federated learning applications in 5G networks, along with their emerging challenges for B5G communications, were discussed in [19]. Categorization and security analysis of federated learning setups were elaborated in [20] while the privacy robustness and efficiency in the presence of an adversarial server was considered in [21]. An extensive investigation on privacy preserving federated learning was carried out in [22] that identified the potential privacy leakage risks in federated learning. However, most of these works are theoretical in nature and do not offer QoS (e.g., bandwidth and delay improvement) along with the implied privacy that are required for B5G networks as explored in the coauthor’s earlier work in [23] by leveraging the theoretical findings on customizing federated learning with asynchronous temporal features in [24]. Based on these recent advancement of federated learning and their various variants, a taxonomy is presented in Table 1 to demonstrate the suitability of federated learning frameworks in B5G networks.

Table 1

A taxonomy of related research work on federated learning techniques and scenarios depicting their suitability in B5G networks.

Reference	Federated learning technique and scenario	B5G-ready?
[8], [12], [14], [18], [20], [22]	Horizontal, vertical, and hybrid data partitioning and/or cryptographic, perturbative, and anonymization techniques for privacy	No
[9], [11], [13], [15], [16], [17], [21]	Specific focus on privacy-preservation in federated learning setups without any wireless communication parameter	No
[22]	Privacy leakage scenarios with external and external attackers, active/passive attacks, inference attacks	No
[24]	Temporally asynchronous weight update of federated learning	No
[19], [23]	Federated machine learning to address energy, bandwidth, delay and data privacy concerns in wireless communications by performing decentralized model training	Yes

Next, we review the existing federated learning applications, with a focus on combating COVID-19. Indeed, federated learning has been conceptualized to revolutionize the smart healthcare landscape recently [25] due to its potential in bypassing the need for centralizing or releasing patient data and raising regulatory, ethical, legal, and even technical challenges. However, the application of federated learning for combating COVID-19 was not considered in this work. The work in [26] raised the importance of respecting and protecting patients’ privacy while collecting their health data for COVID-19 identification. Their work used federated learning to train COVID-19 models and compared performances with popular AI models, including ResNet18, MobileNet, and COVID-Net. These models were also compared with a similar federated learning model presented in [27], focusing on chest X-ray and CT (computed tomography) scan images with normal and COVID-19 samples. A recent endeavor from NVIDIA attempted to construct AI models to predict oxygen needs without the need to share data and devise a generalizable model regardless of geographical location, patient population or data size [28]. The work in [29] identified the generalizability issue in building AI models to gain trust from both patients and clinicians and implemented the unified CT-COVID AI diagnostic initiative (UCADI) framework for federated learning. This initiative allows any research institution or hospital with data and computational resources to contribute their local model parameters to enrich the centralized learning framework’s performance. In this way, the data heterogeneity problem, a key challenge in AI diagnostics in terms of patient diversity, diagnostic image resolution, number of cases, and so forth, is also implicitly considered. In this way, the participants worldwide using UCADI are anticipated to assist health practitioners in detecting COVID-19 patients robustly and reliably. However, the aforementioned methods share an inherent shortcoming in both the system model used and the manner in which the AI models are shared, which restrict the performance of the federated learning technique. For instance, using these methods, the patients may not be able to preserve their privacy because of the need to explicitly share their raw data with hospitals or clinics, which are assumed to secure and perform the federated learning on behalf of the patients. On the other hand, the distributed/transfer learning methods [30] also lead to an exchange of content-specific data (i.e., health data) since they require the UEs to collaborate. Furthermore, the existing applications of federated learning to address the COVID-19 pandemic are yet to explore whether an asynchronous model updating paradigm could be used to boost the learning performance with much-improved network overhead. To mitigate the UEs’ privacy concern and significantly reduce network overhead, a novel proactive method is urgently required in aerial as well as terrestrial base stations.

Considered system model and problem formulation

Fig. 1 summarizes our considered B5G network scenario, which comprises an integrated aerial and terrestrial network. Several UAVs act as aerial base stations to complement the terrestrial base stations (gNBs) to collaboratively serve user equipment/entities or UEs with an improved service coverage. Typically the UEs represent mobile users such as patients with smartphones, wearables, fitness trackers, and other smart devices monitoring health. For simplicity, in the remainder of the paper, the term UE will refer to user-smartphones capable of training local AI models and establishing an uplink with the aerial/terrestrial base stations to exchange the AI model parameters. Further exploiting this scenario and based on our earlier work in [31], the system model is envisaged as a software defined network (SDN) which manages the data, control, and application planes, as depicted in Fig. 2. Using this approach, the SDN controller may virtualize network management algorithms as applications on commodity hardware having various computing resources, such as centralized processing units (CPUs), graphics processing units (GPUs), Baseband Unit (BBUs) pool, and other computing resources to carry out the computation of routing, link scheduling, and other network management tasks for terrestrial and aerial base stations. To summarize, this system model is considered to efficiently manage the integrated aerial and terrestrial links. The SDN can manage the deployment, link quality, and topologies of the UAVs to permit them to adaptively operate in heterogeneous radio access technology (RAT) environments. Thus, the network management is quite flexible and can be easily upgraded simply by updating the corresponding applications. Furthermore, to enhance the usage of the computing resources, optimal caching placement can be done on the drones to store AI models received from many UEs and carry out their own learning to collaborate with other UAVs in learning pandemic features, and thus provide an agile pandemic recommendation and response network.

Fig. 1

Considered UAV-based B5G pandemic response network architecture exploiting heterogeneous B5G network links and federated learning.

Fig. 2

Proposed system model illustrating the B5G network data, control, and application planes.

The considered communication system model is delineated from hereon based on  [32]. During every transmission time round, UE-side local training model parameters are transmitted to the centralized aggregator via terrestrial base stations and/or UAVs to converge to an accurate AI model in a distributed manner. The improved AI model is then periodically shared with the UEs. For each time round, uplink and downlink transmission overheads considering channels with Additive White Gaussian Noise (AWGN) are assumed [33], [34]. The communication rounds are assumed to continue until the global model converges with a reasonable performance measure. Based on the radiograph images collected, the most relevant images are selected by the centralized aggregator for caching at the serving terrestrial base stations and UAVs. Let the UE, terrestrial base station, and UAVs represent the distributed or local computing nodes, referred to as local nodes (LNs) for brevity. Let denote the transmission rate of , which can be estimated as: where , , , and denote the transmission bandwidth of the serving LN, transmission power of , channel gain of , and background noise power, respectively. We assume the same radiograph data-size, for all UEs. Then, the transmission time, , of a local model update with this data-size is expressed as: Then, the time taken for one global iteration by , denoted by is given by: where refers to the local accuracy while denotes the computation time of a local iteration which will be described later. The caching entities of LNs are assumed to have finite storage. The LNs use their local learning models to make intelligent decisions on which radiographs should be cached. The cache is assumed to be able to store files of a constant size. Based on its local data, each LN is assumed to be able to independently calculate an update to the current global learning model and exchange the update to the centralized aggregator to aggregate a new global model. Each UE also trains a local dataset, which is generated from the radiographs, and can be co-trained with other structured data including different current activities, current location, contextual information derived from personal settings of mobile device sensors, and so forth. It is worth noting that each LN, particularly each UE, may have an individual context space. The local learning model in each UE and LN is based on Convolutional Neural Network (CNN) which will be described in Section 4. The local learning model update of is impacted by the quality of its locally available data that can be expressed as the local accuracy, . If the value of is high, we need to perform fewer local (as well as global) iterations bounded by . This can be used to predict the content popularity more precisely and quickly. Now, we describe the computation time of a local iteration, denoted by , in the local model training for as follows: where denotes the size of the local data of . The CPU cycle and number of CPU cycles needed to conduct the local model training with the sample-size are denoted by and , respectively.

In the scenario presented in the preceding section, the small cell UEs, which are served by a terrestrial or aerial base station, participate in pandemic learning such as COVID-19 data collection regarding the user’s symptoms and vitals (e.g., temperature, heartbeat, blood oxygen saturation), travel history (e.g., through geolocation sensing), radiograph monitoring using emerging portable devices, and so forth. The users should not provide raw data to the aerial base stations or the cloud to avoid potential privacy breaches. In addition, the aerial base stations, despite their potential to extend B5G network coverage, are particularly useful for remote/rural locations and pandemic-affected sites where small cells are already prone to experiencing network congestion. Let us consider only those UAV nodes with sufficient energy budget during their flight. With the aforementioned scenario, the terrestrial and UAV base stations require a distributed methodology to learn pandemic features (e.g., COVID-19 symptoms, pneumonia classification using radiographs, and so forth) with privacy preservation to maximize the learning accuracy. Because the base stations lack a global view of UEs in various small cells, how they may leverage the proposed system model in Fig. 2 to perform decentralized machine learning without collecting the raw health data of the users arises a challenge. In this vein, the base stations collect the local training model from the UEs, and share this information with other base stations and then the cloud to further boost the decentralized training among UAVs. The cloud, upon receiving the locally trained models at the UE layer as well as the UAV layer, can then train a global model to arrive at a reasonable decision with significantly high accuracy. In other words, this is a two-layer federated learning problem. The objective is to minimize a loss function: where , , and denote the loss function of , the sample size (locally monitored/acquired health data) of and the training samples used by the cloud. Every UE consists of a weight vector, , which represents its locally trained AI model parameters using and . The value of increases with growing prediction error . Therefore, the constraint is needed to guarantee the convergence of global learning such that all UEs, UAVs, and the cloud can eventually derive the same AI model without explicitly exchanging the raw health data of UEs. Furthermore, in the context of the pandemic, there are two types of UEs considered in the considered system model, namely the individual UE type and the organizational UE type. The individual UEs do not have the capability of collecting certain information (e.g., X-ray images as mentioned) with their simple device type, and in this case the collected information constitutes their location, movement, body temperature, blood-oxygen saturation, and so forth. On the other hand, the organizational UEs such as a caregiver’s system (e.g., a hospital EMR system) or biomedical equipment with IoT capability are assumed to have the capability of sensing/acquiring other information such as X-ray images as presented in the performance evaluation scenario.

To summarize, the research challenge is to design a technique so that (1) each UE is capable of training a shared global model with its local model update based on its data, and (2) each UE shares its updated local model with the cloud through the aerial base stations to revise/update the global model. Similarly, the aerial base stations confront their own local model training problem to predict UEs’ pandemic patterns and participate in second-level federated learning with the global model. Until the loss function is minimized and/or the global model accuracy is deemed reasonable, this distributed training process needs to continue. Therefore, how to design such a decentralized technique with reduced network overhead and privacy-preservation property can be regarded as the main research problem of our work in this paper.

Proposed asynchronous updating federated learning for pandemic response network

The fundamental idea of our proposal is to allow the UEs, constrained by small computational capacity, to train and build localized AI models based on the monitored health data of the user. The UEs participate in the first layer of federated learning to make this possible. Furthermore, the AI model parameter exchange of our proposed system, as shown in Fig. 2, orchestrates a proactive pandemic feature learning algorithm leveraging a second-layer federated learning with the collaboration of UAVs to acquire pandemic model-specific information from the UEs of small cells when the terrestrial base stations are either congested or unable to provide service coverage to UEs due to geographical barriers (e.g., in rural/remote locations). To discover the hidden features from complex pandemic/health data, we utilize a customized convolutional neural network (CNN) to build the AI model at UEs, UAVs, and the cloud [35], [36]. For the UE-generated local AI models, shallow and dense layers are considered to be smartly scheduled for transmission to the serving aerial base station to reduce network overhead. This approach is different from the traditional federated averaging [7], where the parameters of the entire deep neural network are updated at a single shot. This traditional model update process results in large communication overheads. On the other hand, an asynchronous parameter update methodology, inspired by the work in [30], [37] may deal with this problem as follows. The shallow layer (general) parameters acquire general features of the pandemic while the deep layer parameters, big in size, capture specific features regarding specific symptoms and context information of the users. Consequently, the shallow layer weights can be updated and exchanged more often compared to deep layer-parameters. This asynchronous weight updating methodology is shown in the algorithmic steps of Fig. 3.

Fig. 3

Proposed algorithm.

First, we discuss the local model update at in Fig. 3. The proposed algorithm has three inputs: the UE identifier , the weight vector of the UE-generated model , and a control flag . The collected health data of the pandemic and context information are split into mini-batches, and represents the mini-batch size. denotes the epoch of the local model. The control flag is exploited to select whether all the layers or the shallow layers will be updated. Stochastic Gradient Descent (SGD) is used . Here, and refer to the learning rate and minibatch, respectively. On the other hand, the aerial base station, , polls the UEs by initializing a clock, in synchronization with the UEs and the cloud. receives the timestamp for each of the UEs it serves. Then, starts a temporary buffer. Using the timestamp information, makes a decision to execute either the shallow or deep parameters update. Then, for each round, the UAV polls its served UEs ( to where is an index between 1 and ) and receives the relevant parameters from given the current timestamp and the control variable. Next, appends the parameters received from to the buffer. After polling all the UEs, this buffer contains the information of shallow-only or complete weight vector parameters for the UEs served by that UAV. As shown in Fig. 3, the shallow-only/deep parameters are sent during the flag switch interval, which is considered to be much smaller than the updated global model receiving interval of each UE. The content of the buffer is then sent to the cloud. also locally executes its deep learning structure for predicting pandemic trends in collaboration with the neighboring UAVs. Then, the UAV continues to poll the UAVs again in the same manner until the model accuracy becomes reasonable.

At the cloud-end, the temporally weighted aggregation occurs over multiple communication rounds [37]. The timestamps for general and specific weights/parameters are initialized. The federated learning takes place in each round , which is split into time-slots. Before each round, the cloud exchanges the timestamp list it initialized for each UE to the serving base station. Then, for each round , the time-slots which are in (the set of time-slots during which specific parameters are fetched from UEs), the control variable is set to true. A participating subset of the clients is randomly selected per round. The specific and general parameters are fetched from each base station. Thus, the aggregation is performed to update the shallow and deep parameters ( and , respectively).

Based on these steps, we present the two algorithms for our envisioned asynchronous federated learning model for privacy-preserving COVID-19 screening, namely Algorithms 1 and 2. The global and local AI models, constructed at the cloud and aerial base station/terrestrial base station/UE, respectively, are facilitated by either a custom ANN (artificial neural network) model or a customized CNN model developed in our earlier work to carry out a centralized COVID-19 prediction [38].

First, we describe Algorithm 1 used for training the initial global AI model. This algorithm accepts the set of all available UAVs, terrestrial small cell base stations (TSCs), and UEs. The objective of the algorithm is to return , the trained global model. The algorithm parameters are initialized in step 1. The elements of the set of all the local or distributed nodes (i.e., the distributed learning agents in terms of the deployed UAVs, TSCs, and UEs), referred to as , is iterated through steps 3 to 5. Thus, the parameters obtained from the distributed nodes can be aggregated. The aggregated parameters are then shared with the distributed nodes by the cloud. Learning continues until the learning performance in terms of loss is improved compared to a pre-determined threshold, (), in step 7 of Algorithm 1.

Next, Algorithm 2 is presented to execute the distributed learning on the LN’s side. The inputs of the algorithm consist of , , , and . Throughout the first three steps, the required parameters related to the algorithm and the local model () is set to selected values. Following this, a binary parameter, referred to as the contextual switch and denoted by , is set to false. In the subsequent step, the data size of each radiograph data is evaluated by setting the instance and feature sizes as and , respectively. Then, the new data is loaded and standardized as: where indicates the data after performing resizing, denotes the mean value of the data, and refers to the standard deviation.

Step 9 of Algorithm 2 signifies that the training process at each LN takes place over to iterations, where denotes the number of time-steps during which the whole process at each LN is executed once. After time rounds, when the condition for setting the switches on is verified in step 10, the contextual switch () is set to true. Then, an alert is triggered to activate an optional, physical switch. Next, in step 14, the data access information for LN is stored in a dedicated block (). The local model is trained by adopting the selected hyperparameters in step 15 of Algorithm 2. Steps 16 to 21 are executed while flag is true. Through these steps, the local weight () with shallow parameters and relevant access and time-step information (, ) are shared with the cloud. The parameter refers to the rate of the deep parameters exchange. The parameter holds the iteration information when the deep parameter exchange with the cloud occurs. Through steps 20 and 21, is set to false. This is done to avoid unauthorized transmissions. Furthermore, an on-display alert is generated at the LN for the LN manager to turn off the physical switch until the next time round commences. If is false, the shallow parameters of ratio from the local model () are shared with the cloud. On completion of the training over time-steps, the already employed data () is permanently removed from the cache to prevent possible breaches of user data in the future.

From hereon, we analyze the convergence performance of the proposed asynchronously updating federated learning to address the original optimization problem. The centralized aggregator transmits the model parameters to LNs to initialize and train their respective localized AI models. The UEs, TSCs, and UAVs selectively exchange shallow-only and deep parameters using Algorithm 2 such that the update of each ’s local model parameter depends on the global model while the update of the global model relies on all LNs’ local federated learning models. The update of the local model depends on the learning algorithm such as Gradient Descent, Stochastic Gradient Descent (SGD), or Randomized Coordinate Descent (RCD). The update of the global model is obtained as . The update of ’s local model at time is expressed as: where denotes the learning rate and represents the gradient of with respect to .

Assuming that and , the update of global model at time can be given by:

Here, , where is 1 if a resource block is assigned to for transmitting its local model parameters with sufficient transmit-energy. Then based on the work [39], the federated learning algorithm converges to an optimal global model after the learning steps in Algorithm 2. Interested readers may view the proof of the expected convergence rate in [39].

Performance evaluation

To synthesize the dataset required for experiments, DJI Matrice 300 RTK drone with URUAV GRAPHENE 6S 22.2 V 6000 mAh 100C Lipo Battery XT90 [40] having an energy capacity of 133.2 Wh has been considered in this work. This battery is commonly used for heavy-duty tasks such as local AI model training. The drone has a flight time of 15 min which is considered enough to offload information from mobile UEs not having access to TSCs due to lack of infrastructures or overburdened TSCs. Up to 10 drones are considered in the simulation. In other words, the number of LNs is varied between 2 to 10 in the conducted simulations.

To demonstrate the performance of our envisioned asynchronously updating federated learning model, a public repository of X-ray images with COVID-19, pneumonia, and normal chest X-ray images is employed. Compared to the existing work dealing with small datasets, we prepared a large and robust dataset by merging four different existing datasets consisting of Posteroanterior (PA) chest X-rays based on the coauthors’ earlier work in [38]. It is worth noting that the hyperparameter tuning of the underlying ANN and CNN models was done in our earlier work [38]. As for the hyper parameter tuning of our presented federated learning setup, the tuning of deep/shallow parameter exchange ratio/interval is considered for hyperparameter optimization. According to the underlying AI model, two variations of the proposed asynchronously updating federated learning models are simulated based on ANN and CNN, which we refer to as Fed-ANN and Fed-CNN, respectively, for clarity. Furthermore, we define the iteration and time round for evaluating the performance of our proposal. An iteration refers to the number of times the whole process runs (e.g., the local training for the LNs and exchanging parameters). On the other hand, a time round consists of multiple iterations, after which the cloud is updated with the deep weights. In other words, an iteration is an atomic unit relative to the time round. For instance, if the number of iterations is 20 and the number of time rounds is 5, then for the 5th, 10th, 15th, and 20th iteration, the deep model update is triggered in our proposed approach while the shallow model parameter update takes place during the other iterations. The number of iterations was set to 20, considering the learning loss converged in all considered cases. For all the simulations, the epoch-size and batch-size were set to 50 and 16, respectively. The exponential linear unit (ELU) was employed as the activation function, while Adagrad was used as the optimizer to select and regularly update a preconditioned stochastic gradient descent smartly. For the Fed-ANN model, a fully connected layer followed by the output layer was considered. On the other hand, the Fed-CNN model consists of three convolution layers, followed by three fully connected layers and the output layer.

Table 2 demonstrates the comparison of Fed-ANN and Fed-CNN models for 2 to 10 LNs over varying time rounds during the learning phase. The Fed-ANN’s best accuracy is 92.88%, which takes 27 iterations to converge, requiring 15 time rounds. On the other hand, Fed-CNN requires 50 iterations and yields a higher COVID-19 prediction accuracy (94.83%), consuming only 10 time rounds.

Table 2

Performance comparison of federated learning architectures for a varying number of users and time rounds (learning phase).

Number of LNs	Fed-ANN				Fed-CNN
	Time round	Iterations to converge	Accuracy	Loss	Time round	Iterations to converge	Accuracy	Loss
2	15	6	0.8968	0.0629	10	6	0.9477	0.0293
4	10	10	0.9038	0.0599	10	16	0.9456	0.0314
6	10	16	0.9157	0.0561	5	27	0.9468	0.0309
8	10	26	0.9208	0.0527	10	31	0.9444	0.03201
10	15	27	0.9288	0.0507	10	50	0.9483	0.02901

Fig. 4 compares the learning accuracy and loss of the Fed-ANN and Fed-CNN architectures over different time rounds varying from 5 to 20. Notice from Fig. 4a that both models’ learning accuracy is over 90% at the fifth time round. Both the variations of our proposal gain higher accuracies with the increasing number of time rounds. However, the Fed-CNN model exhibits superior performance for all the time rounds. This is also illustrated in learning loss, in terms of the mean squared error (MSE), in Fig. 4b.

Fig. 4

Learning accuracy and loss comparison of adopted federated learning architectures over different time rounds.

Next, the impact of iterations on the learning loss performances of Fed-ANN and Fed-CNN is investigated in Fig. 5. Both Fed-ANN and Fed-CNN demonstrate improvement in the learning with an increasing number of iterations. However, the Fed-CNN model shows a much faster convergence trend, indicating its viability as the candidate deep learning technique for the proposed federated learning framework.

Fig. 5

The value of the loss function over growing number of iterations for the ANN and CNN-based federated learning architectures.

In Fig. 6, we compare the required execution time and memory requirement for the training phase for the proposed Fed-ANN and Fed-CNN models. While these overheads incurred by Fed-ANN are lower than that of the Fed-CNN model during training, the accuracies in the testing phase between these models as well as the centralized benchmark algorithm [38] are found to be quite similar, as demonstrated in Table 3. Thus, there is a slight tradeoff between the accuracy and overhead between our proposal’s two variations (i.e., Fed-ANN and Fed-CNN), which demonstrates Fed-CNN as the most suitable technique for the COVID-19 detection task.

Fig. 6

Required execution time and memory consumption comparison (Learning phase).

Table 3

Performance comparison among diverse AI methods (testing phase).

	Accuracy	Precision	Recall	F1-score
Centralized deep learning benchmark	0.94630	0.94513	0.94352	0.94433
Fed-ANN	0.91914	0.91763	0.91914	0.91419
Fed-CNN	0.93891	0.93773	0.93891	0.93557

Finally, Fig. 7 shows how our proposed asynchronous weight update method reduces the network overhead. In this case, 20 rounds are considered, which are enough to demonstrate the overhead reduction performance. Notice that with the larger deep parameter rates, a more overhead reduction is possible by performing more generic parameter exchanges from the local model using the LN to the cloud and only transferring the specific parameters after a relatively high number of time rounds.

Fig. 7

Overhead reduction for the federated learning asynchronous update method over different time rounds and deep parameter exchange ratios.

Future directions and open issues

In this section, we highlight the open issues and the benefits that the proposed approach has in the B5G context, and then describe the future directions to introduce further security/privacy augmentations that were beyond the scope of the current work.

As discussed in Section 2, only few federated learning techniques started to focus on considering the QoS improvement issue along with privacy-preserving federated learning that is critical for B5G network users in UAV-served tiny cells where competition for bandwidth will be immense despite the high bandwidth capacity. The proposed work provides a proof-of-concept demonstrating how asynchronously updating weights of the locally trained deep and shallow models can be efficiently scheduled to significantly reduce the overhead reduction while attaining reasonable execution time and memory use as well as achieving high learning accuracy. This is particularly important for emerging health applications for distributed processing and analytics on users’ electronic health records, locations, and biomedical image data as demonstrated in the COVID-19 usecase in this paper. The opportunistic use of the temporal synchronization in a smart manner, thus, enlarges the scale of training data with resource-constrained and highly mobile B5G mobile under dynamically changing RAT conditions, and at the same time, protects the privacy of the user data.

As an open research issue, we provide a theoretical argumentation, in a high-level manner, describing possible integration of the Blockchain technology and/or methods to enable Blockchain-based applications with the proposed asynchronously updating federated learning framework. This is due to the fact that the security of local parameters, the learning quality, and the varying computing and communication resources, remain unexplored topics in the existing federated learning schemes [41] including our proposal. Blockchain is an advanced technology which has modified the ways to store, exchange, and record the data using a public ledger so that the transactions are immutable [42], [43], [44]. On the other hand, the aim of federated learning is somewhat limited to applications orchestrated by the centralized aggregator as demonstrated in Algorithm 1. Hence, the robustness of the centralized aggregator relies upon the secrecy performance and resiliency of the central server. In other words, the aggregator may act as a single-point-of-failure by jeopardizing the entire federated learning framework [45]. Furthermore, despite the local data from LNs not being shared explicitly in their original format, knowledgeable adversaries may approximate the raw data reconstruction during the aggregation process [45]. Thus, privacy leakage may occur during model aggregating by outsider threats [46]. Also, concealing the COVID-19 training data may present the malicious users with an opportunity to inject backdoors into the trained model [45], [47]. Researchers are recently considering adversarial models whereby an attacker may be able to inject backdoors to the trained model during federated learning, and then can leverage the backdoor to make the model misclassify at a later time. In the future, how more seamless integration between asynchronous federated learning and blockchain should be integrated needs to be carefully investigated to avoid the aforementioned open research issues. Furthermore, this work can be considered as proof-of-concept and can be extended toward other pandemic monitoring given that the dataset is available from public health agencies and other relevant organizations at the national level.

Next, because of the heterogeneous resources and capabilities of different RATs and UEs in B5G networks, the selection of participating nodes (e.g., a mobile UE as opposed to a more powerful UAV node or a even more powerful terrestrial gNB) may have a dominating impact on the performance and convergence-time of federated learning. Furthermore, in the global model aggregator/server-side, initialization level of the global model and fine-tuning parameters such as the duration for each iteration, as well as duration for local model exchange for deep/shallow parameters, need to be taken into account in the future.

Conclusion

In this paper, we highlighted the importance of formulating an intelligent pandemic response network, particularly during the ongoing pandemic of COVID-19. We presented a drone-based heterogeneous computing system model to support the patient/user entities or user equipment (UEs), which have the capability of collecting their health data but are not bound to sharing the collected data with the drones/other base stations to be delivered to the central cloud managed by the public health authority and/or other hospitals. Instead, they share the wisdom with the network entities in more frequent, small-sized locally trained shallow model parameters and infrequent, heavy deep, or specific model parameters regarding their monitored health data. On the other hand, the UAVs poll each UE systematically, exchange them further with other UAVs, and forward the models to the cloud. They also help, in addition to terrestrial base stations, to deliver the updated global model to the UEs. Using this smart technique, the user data’s privacy is inherently respected, and the network overhead is significantly reduced. Through the performance evaluation on a particular use-case using X-ray images containing COVID-19, pneumonia, and normal samples, the effectiveness of two variants of our proposed asynchronously updating federated learning approach demonstrated the effectiveness as a viable pandemic response network methodology.

CRediT authorship contribution statement

Nidal Nasser: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Writing – original draft. Zubair Md Fadlullah: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Writing – original draft. Mostafa M. Fouda: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Writing – original draft. Asmaa Ali: Conception and design of study, Analysis and/or interpretation of data, Writing – original draft. Muhammad Imran: Conception and design of study, Analysis and/or interpretation of data, Writing – original draft.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.