Continuous Hidden Markov Model Based Spectrum Sensing with Estimated SNR for Cognitive UAV Networks.

In this paper, to enhance the spectrum utilization in cognitive unmanned aerial vehicle networks (CUAVNs), we propose a cooperative spectrum sensing scheme based on a continuous hidden Markov model (CHMM) with a novel signal-to-noise ratio (SNR) estimation method. First, to exploit the Markov property in the spectrum state, we model the spectrum states and the corresponding fusion values as a hidden Markov model. A spectrum prediction is obtained by combining the parameters of CHMM and a preliminary sensing result (obtained from a clustered heterogeneous two-stage-fusion scheme), and this prediction can further guide the sensing detection procedure. Then, we analyze the detection performance of the proposed scheme by deriving its closed-formed expressions. Furthermore, considering imperfect SNR estimation in practical applications, we design a novel SNR estimation scheme which is inspired by the reconstruction of the signal on graphs to enhance the proposed CHMM-based sensing scheme with practical SNR estimation. Simulation results demonstrate the proposed CHMM-based cooperative spectrum sensing scheme outperforms the ones without CHMM, and the CHMM-based sensing scheme with the proposed SNR estimator can outperform the existing algorithm considerably.

1. Introduction

With the advantage of high flexibility and low deployment cost, unmanned aerial vehicles (UAVs) have been widely used in military communications, weather monitoring, emergency rescue [1] and some other UAV-assisted Internet of Things (IoT) applications [2]. The large-scale deployment of UAVs has exacerbated the shortage of spectrum resources. However, the existing spectrum allocation strategies cannot effectively use the scarce spectrum resources, which becomes the bottleneck for enhancing the communication performance of UAVs [3]. Cognitive radio (CR) is proposed to solve the problem, which improves the spectrum efficiency by perceiving spectrum holes and providing secondary UAVs with opportunities to reuse idle spectrum. Thus, CR can further guide the spectrum utilization in cognitive unmanned aerial vehicle networks (CUAVNs), including the resource allocation for low-latency communications [4], high-quality services with limited resources [5], maximum achievable throughput [6], and optimal power allocation. To enable CUAVNs, accurate spectrum sensing attaches great importance.

With the development of UAVs, spectrum sensing in unmanned aerial vehicle networks (UAVNs) has attracted attention from both academia and industry. The detection performance is enhanced by using multiple secondary UAVs in [7,8]. Authors in [3,9] consider the combination of UAVs and terrestrial communication equipment for air-ground-integrated spectrum sensing. The heterogeneity of spatial information is further taken into account by using the 3D [10], so as to improve the spectrum sensing detection performance. The above studies introduce information from multiple users and spatial dimensions to improve the spectrum utilization of CUAVNs. However, they do not consider the temporal information of the spectrum, which affects the detection performance of UAVNs. Exploring the temporal correlation of spectrum states by focusing on the Markov property is an effective and novel idea to enhance the detection probability [11].

Another concern is that knowledge of the signal-to-noise ratio (SNR) is required before spectrum sensing [12], e.g., energy detection and cyclostationary feature detection. Therein, it is assumed that SNR is perfectly known. However, UAVs need to move across a large area in lots of applications, and the links between two UAVs that are far apart frequently break and reestablish [13], which degrades the sensing performance and the spectrum utilization. Besides, large flying areas, unstable links, and dynamic network topologies lead to variable SNRs in CUAVNs, which makes the assumption of pre-known SNRs no longer applicable, bringing new challenges for sensing in UAVNs.

To overcome these problems, we propose a continuous hidden Markov model (CHMM) based sensing scheme with a novel space smoothing second- and fourth-order moments (SS-M2M4) SNR estimator. Note that the true states of the unauthorized spectrum are not observable, however, the sensing results originating from the unobservable spectrum states can be easily obtained. Therefore, a hidden Markov chain model fits well with the spectrum sensing procedure. To fully exploit the temporal information and further enhance the detection performance, we combine the spectrum sensing scheme with CHMM. To the best of our knowledge, our work is the first to consider the continuous hidden Markov model in non-centralized CUANs. Moreover, we further provide theoretical analysis of the detection probability and the false alarm probability, while previous works mainly focus on numerical simulations. Besides, for generalized estimation and higher accuracy, based on a general SNR estimation method of a second- and fourth- moments (M2M4) estimator [14], we propose a SS-M2M4 SNR estimator. The SNR smoothness of the neighboring CUAVs has been taken into account, enlightened by the smoothness of the graph signal [15]. Compared to the widely-used M2M4 estimator, our proposed SNR estimator achieves more accurate estimation. With the SNR provided by the proposed estimator, our CHMM-based spectrum sensing scheme can achieve a high detection probability. Our contributions are summarized as follows:

We propose a spectrum sensing scheme based on a continuous hidden Markov model in CUAVNs to obtain better sensing performance.

We derive the closed-form detection probability and false alarm probability expressions of the proposed CHMM-based spectrum sensing.

Considering the fading similarity within the neighboring CUAVs in practical applications, we propose an SNR estimation scheme based on signal smoothness, which reduces the SNR estimation error effectively, and further enhances the performance of CHMM-based cooperative spectrum sensing.

The remainder of this paper is organized as follows. Section 2 discusses the related work. Section 3 introduces the system model. In Section 4, the CHMM based spectrum sensing scheme is proposed. In Section 5, the closed-form detection probability and false alarm probability expressions of the proposed method are derived. In Section 6, we design a novel SNR estimator to enhance the proposed CHMM-based spectrum sensing scheme with a more accurate SNR. The simulation results are presented to evaluate the proposed algorithms in Section 7. Section 8 concludes the work and discusses the possible future work.

2. Related Work

2.1. Hidden Markov Model (HMM) Based Spectrum Sensing

Hidden Markov model (HMM) based spectrum sensing means using the HMM to model the spectrum sensing procedure. As the HMM has a hidden layer and an observable layer, it fits well with the sensing procedure whose spectrum states are unknown but the receiver can be obtained. Exploring the Markov property of the spectrum states is an effective way to enhance the sensing performance [11]. Compared with deep learning based spectrum sensing [16,17], CHMM-based spectrum sensing has a stronger interpretability and smaller delay, and the initial probability distribution can be used to calculate the initial spectrum utilization. In addition, the obtained parameters can also be used for digital twinning of the communication system. It has been validated that the spectrum state can be modeled as a Markov chain by analyzing real-world measurements [18]. A hidden Markov model based scheme [19] is proposed to predict the arrival of the primary user (PU). Authors in [20] evaluate the reliability of HMM based cooperative spectrum sensing in cognitive radio networks, in the presence of random malfunctioning of secondary user nodes participating in the process. Occupancy prediction schemes based on a discrete hidden Markov model (DHMM) and a continuous hidden Markov model are investigated in [21,22,23,24], respectively. Authors in [21,22] adopt DHMM to model the spectrum sensing procedure, which do not make full use of the information obtained by the receiver. Authors in [23,24] use CHMM to model the sensing procedure of CUAVNs. However, they do not consider the dynamicity of UAVs, which is an important characteristic of the UAV networks [25]. Besides, centralized spectrum sensing methods in [23,24] do not work well in CUAVNs since the global fusion center is usually unreachable to secondary CUAVs. Thus, in this paper, we propose a CHMM-based spectrum sensing method to enhance the spectrum utilization in CUANs.

2.2. SNR Estimation

SNR estimation refers to the calculation of SNR by using signal information [14]. Various algorithms require SNR estimation for optimal performance if the SNR is not constant, such as linear diversity combining techniques and Viterbi algorithms with soft-decision [14]. Note that the knowledge of SNR is also required for typical commonly-used spectrum sensing [12]. However, in practical UAV applications, it is difficult to obtain accurate SNR. Conventionally, SNR estimators require knowledge of the signal or the channel, such as the maximum likelihood (ML) SNR estimator [26] and particle swarm optimization (PSO) SNR estimator, based on parameters of hardwares or channels [27]. In addition, there exist estimators designed for specific signals or specific feature spectrum sensing methods, such as the SNR estimator [28] for M-ary amplitude phase shift keying (M-APSK) modulated signals, the SNR estimator for signal with Polar code [29], and the estimator for eigenvalue-based spectrum detectors [30]. Lacking the prior knowledge of the signal/channel and design for specific signals/sensing methods make it hard to generally adopt the above estimators to various CUAVNs. Besides, some scholars have paid attention to deep learning based SNR estimation methods, such as the convolutional neural networks (CNN)—long short term memory (LSTM) based SNR estimators [31] and the CNN-based SNR estimators designed for UAVNs [32]. However, in the spectrum sensing of CUAVNs, the deep learning based methods are too complicated and take more time, leading to the sensing term being missed. Taking all the above into consideration, we design a generalized and low-complexity SNR estimator named SS-M2M4. Based on the proposed SNR estimator, the CHMM-based spectrum sensing method can further enhance detection performance.

3. System Model

In order to detect whether the authorized spectrum of PU is occupied, a spectrum sensing method is adopted by multiple UAVs [3,33]. In this paper, we propose a CHMM-based spectrum sensing method with consideration of the imperfect SNR estimation to enhance the detection probability. The overview of this unified scheme is shown in Figure 1. First, to obtain the single-time spectrum sensing fusion result, we adopt the max-min distance clustering algorithm [34] and heterogeneous two-stage-fusion spectrum sensing scheme [35], which are described in Section 3.1 and Section 3.2, respectively. Then, to fully take advantage of the temporal correlation of the spectrum states, we propose a CHMM-based spectrum sensing method, which will be introduced in Section 4. Considering the imperfect SNR estimation in practical applications, we propose a novel SNR estimator shown in Section 6 to offer the sensing scheme more desirable SNRs. Finally, combining the CHMM-based spectrum sensing and the proposed SNR estimator, we propose the unified CHMM-based spectrum sensing scheme with advanced SNR estimator.

Figure 1

The unified framework of the CHMM-based spectrum sensing scheme with the SS-M2M4 estimator.

3.1. Clustering Method

Considering the fusion delay brought by distributed cooperative spectrum sensing (DCSS), and to further provide the proposed SS-M2M4 SNR estimator with graph topology, a max-min distance clustering algorithm [34] is adopted. It divides CUAVs with similar position and mobility into the same cluster, and selects the one with the highest trust value in each cluster as the cluster head. Thus, we can adopt intra-cluster centralized cooperative spectrum sensing. Due to the similar locations, the UAVs in the same cluster have similar SNRs, which facilitates the SNR estimation. The cluster heads are less than the number of the CUAVs, so the fusion delay will be reduced and, therefore, we adopt inter-cluster distributed cooperative spectrum sensing.

3.2. Heterogeneous Two-Stage-Fusion Spectrum Sensing

To further improve the sensing performance in CUAVNs, a heterogeneous cooperative spectrum sensing scheme is employed with the clustering outcome, as shown in Figure 2. A symbol table of the notations is shown in Table 1.

Figure 2

Heterogeneous Two-Stage-Fusion.

Table 1

Notations.

Notation	Description	Notatiom	Description
a	the amplitude of the transmitted signal	T_i	the likelihood ratio test statistic of the ith cluster
f_c	the carrier frequency of the transmitted signal	T_ij	the cyclostationary detection statistics or energy detection statistics
φ	the carrier phase offset of the transmitted signal	ω_ij	weight of T_E_ij
M	the number of sampling point	ρ_i	weight of T_C_i
α	the cyclic frequency of the received signal	γ_C_i	the SNR of the ith cluster head
T_C	the test statistic for the first-order cyclostationary detection	γ_E_ij	the SNR of the jth node in the ith cluster
T_E	the test statistic for energy detector	L	the Laplacian matrix of the cluster heads topology
T_ic	the statistics of the ith cluster	α_l	step size
T_C_i	the cyclostationary detection statistics of the ith cluster head	d	maximum node degree of the cluster graph
T_E_ij	the energy detection statistics of the jth the node in the ith cluster	δ_h_i	the weight of the ith cluster head
K	the number of CUAVs in the ith cluster	T^*	the final consensus
K_E	the number of cluster members in the ith cluster	W	the consensus weight matrix

3.2.1. Heterogeneous Spectrum Sensing Based on Clustering

To be specific, in the sensing state, a heterogeneous cooperative spectrum sensing scheme [36] is used according to the clustering result. Here, the "heterogeneous” means two different detection schemes: energy detection and cyclostationary detection [36]. Cluster heads adopt cyclostationary detection, while other secondary UAVs in the same cluster adopt energy detection. Cyclostationary feature detection is adopted to the cluster head, since it has great detection accuracy and can maintain good performance even in environments with low SNR. The cluster members (CMs) adopt energy detection, since it is simple to implement and does not require prior knowledge of the channels. Similarly to [36], we assume that the transmitted signal of the primary user is a sinusoidal signal, in which the n-th sample is expressed as where a is the amplitude, is the carrier frequency, and is the carrier phase offset. The received signal of the cluster heads is written as where denotes the additive Gaussian noise with zero mean and unit variance. indicates that the spectrum is absent, while indicates that the spectrum is occupied. The test statistic for the first-order cyclostationary detection [36] is expressed as where is the cyclic frequency, M denotes the number of sampling point. The received signal of the CUAV cluster member is modeled as the same as

The test statistic for the energy detector [36] is expressed as where M denotes the number of sampling points. Let denotes the vector of the statistics from the ith cluster, is the cyclostationary detection statistics of the ith cluster head, and its distribution [36] can be expressed as where , denotes the SNR of the ith cluster head, and is the folded normal distribution [37]. is the energy detection statistics of the jth the node in the ith cluster. When M is large enough, approximatively follows normal distribution according to the central limit theorem [38], but slightly different to [38], our is 1/M times the energy detection statistics in [38], thus the our SNR is M times the SNR in [38], and we assume the energy of the received signal is 1. Thus, the distribution of can be expressed as where is the SNR of the jth node in the ith cluster.

3.2.2. Two-Stage-Fusion

In the fusion duration, we adopt a two-stage-fusion scheme [34], which includes the intra-cluster fusion stage and the inter-cluster fusion stage. The CUAV cluster members adopt energy detection, and we apply high-accuracy centralized soft fusion in the intra-cluster since the performance of energy detection is somewhat not precise. In the intra-cluster fusion stage, according to the assumption that the observations are independent, we can attain the likelihood ratio test (LRT) [36] of the ith cluster there are K CUAVs, CMs in the ith cluster, where . denotes cyclostationary detection statistics or energy detection statistics, and represent the probability density under hypotheses and , respectively. According to the , , the LRT in Equation (8) can be simplified [36] as where , and .

As for the inter-cluster fusion stage, considering the good sensing performance of cyclostationary detection and the large distance between the cluster heads, distributed consensus-based fusion [38] is performed. Each cluster head communicates with its neighbouring cluster heads to exchange information, and the exchange process is iteratively done. The initial information (which is the intra-cluster fusion result) of the ith cluster head is denoted as . Then, according to the network topology, these cluster heads repeatedly iterate until covers a common value. The consensus-based scheme [38] is where , and L is the Laplacian matrix of the cluster heads topology. is the step size, and it satisfies , d is maximum node degree of the graph. , where [38] is the weight according to the channel condition of the ith cluster head, and it satisfies . The cluster heads communicate with their own neighbors, then a final consensus [38] is reached as

4. Continuous Hidden Markov Model Based Spectrum Sensing

To fully take advantage of the temporal correlation of the spectrum states, we propose a CHMM-based spectrum sensing scheme as shown in Figure 3. Firstly, hidden HMM and CHMM are introduced, and the suitability regarding the spectrum states as a continuous hidden Markov model are analyzed. Then, the model is trained with the forward-back algorithm and Baum–Welch algorithm. The prediction obtained according to CHMM model is used to assist the sensing.

Figure 3

CHMM-based Spectrum Sensing.

4.1. Hidden Markov Model

The hidden Markov model is a double stochastic process with a hidden layer and an observable layer. The hidden process is an unobservable Markov chain, which can be obtained through the observed states.

The true states of the spectrum are not observable but the sensing results can be easily obtained. Therefore, the hidden Markov chain model fits well with the PU spectrum state. For HMM, there are three basic problems that need to be solved, i.e., the evaluation problem that computes the probability of the observed fusion result sequence, the learning problem that adjusts the model parameters to maximize the probability of the observed sequence, and the predication problem that calculates the most likely hidden spectrum state sequence according to the observation sequence and model parameters.

In order to avoid the distortion caused by the discretization of continuous variables in the cluster heads, we consider the continuous HMM, which replaces the discrete observation states with continuous characteristics. With more specific spectrum information, we can obtain better detection performance.

4.2. Continuous Hidden Markov Model of Spectrum States

The PU spectrum state at time instant t is given by , and it can be 0 or 1, where 0 represents spectrum absence, 1 denotes spectrum occupancy. The sequence of the PU states can be seen as the hidden Markov chain. is the fusion value of the heterogeneous two-fusion-stage spectrum sensing at time instant t, and is the observable layer. The hidden Markov chain and the observable layer constitute a continuous hidden Markov model, which can be formulated as [39], where represents the initial probability vector of the hidden spectrum state, A is the transition matrix of the two states. The continuous hidden Markov model can be represented in Figure 4. are the parameters of the observation probability distribution. The Gaussian mixture model (GMM) is used to model the probability, as the Gaussian process has good adaptability in dealing with complex regression problems and classification. Thus, the observation probability in state i according to the GMM can be written as which is composed of M Gaussian mixtures. i denotes the spectrum state. is the proportion of the mth mixture coefficient in state i, o denotes the fusion result calculated by the clustered heterogeneous two-fusion-stage scheme, and represent the mean and the covariance of mth mixture in state i, respectively.

Figure 4

Continuous hidden Markov model.

To employ the continuous hidden Markov model to cognitive UAV networks, the forward-backward algorithm and Baum–Welch algorithm are utilized to solve the evaluation problem and the training problem, respectively. As for the prediction problem, the Viterbi algorithm is utilized.

4.3. Evaluation Process and Learning Process of Continuous Hidden Markov Model

For the evaluation problem, the forward-back algorithm is used to calculate the probability of the observed fusion value sequence, and it can be divided into two parts: the forward algorithm and backward algorithm. For a given and the spectrum state at time t, the forward quantity is the joint probability of sequence O from the initial time to time t, and the state in at time t, is the joint probability of sequence from time to the final time and the state in at time t. The forward and backward quantities [39] are defined as with the initializations

The forward and backward quantities can be calculated by

Then, combining the forward and backward algorithm, we can get the forward-backward algorithm, then the probability of the observed fusion result sequence O with the given model parameters is obtained [39] as

To solve the learning problem, the Baum–Welch algorithm is adopted, which is one of the expectation–maximization algorithms and uses the forward-backward algorithm in each expectation process. Before using this algorithm, we need to define three parameters: , and . denotes the probability of the ith spectrum state at time t, denotes the probability that at time t the spectrum state is and at time the spectrum state is , denotes the probability of the mth Gaussian mixture of state at time t. , and can be calculated [39] as follows:

When enough training data are provided, that is, the sequences of fusion results obtained from the heterogeneous two-stage-fusion sensing scheme and corresponding spectrum states, the Baum–Welch algorithm offers a way to train the model, outputting good CHMM parameters. In specific, initial model parameters are first selected according to the spectrum condition. Second, , , , and are calculated. Third, the parameters are updated according to Appendix A. The forward-backward procedure and the updating procedure are repeated until the probability of the observation sequence satisfies the convergence condition or the increments of parameters are less than threshold. Finally, we can obtain the trained model parameters .

4.4. Predication of Spectrum State with CHMM

In this section, we adopt the Viterbi algorithm to solve the predication problem. With the learned model parameters and the observed sequence of fusion results, we can calculate the joint probability of the observed sequence. The real spectrum state sequence is calculated [39] as where denotes the spectrum state at time t, represents the transition probability from state to , denotes the observation probability of , when the real spectrum state is . Then, select the sequence with the maximum probability as the prediction sequence

Next, we can get the prediction and the prediction sequence X, and compare the predication result X with the real state sequence to get the prediction accuracy . When the prediction is “busy”, can denote the probability that the spectrum is really occupied at time T, can denote the probability that the spectrum is really absent at time T. Similarly, when the prediction is “idle”, the probability of occupancy is , and the probability of absence is .

Combining the prediction accuracy of cluster heads with the fusion result of the detectors, we can get a new false alarm probability and detection probability. There are mainly two kinds of predictions: busy and idle. When the prediction is “busy”, we multiply the fusion result T by . Similarly, when the prediction result is “idle”, we multiply the fusion value by . Then we obtain the final fusion statistic adjusted by the prediction. After that, we compare the final fusion statistic with the threshold. When the fusion statistic is larger than the threshold, the decision is “busy” and vice versa. Thus, the final detection probability can be calculated as

5. Analysis of Detection Performance for CHMM-Based Spectrum Sensing

Under means the spectrum is occupied. When the sampling points are large enough, approximately follows normal distribution, as does (which will be explained in Section 7.1). Thus, when the spectrum is occupied, the fusion result can be calculated as where n is the number of clusters, is the weight according to the channel condition of the ith cluster head, is the number of cluster members (CMs) in the ith cluster. According to [36], the weight can be simplified as , and . The fusion result T is an approximately normally distributed random variable with mean and variance , the mean and the variance are expressed as and

Then we can obtain the final detection probability of the CUAVNs, where is the threshold.

Under , when M is large enough, approximately follows normal distribution, and the probability density can be approximately represented as

The probability density of the fusion result T can be represented as where , , , . The proof of is shown in Appendix B.

The false alarm probability can be calculated as follows, where ,

6. CHMM-Based Spectrum Sensing with Practical SNR Estimation

The above work is based on the assumption that the SNR is perfectly known, i.e., we use a perfectly-known SNR when calculating the fusion weight, detection probability and false alarm probability. In this section, we further consider the scenario that the SNR is not perfectly known, and design a novel SNR estimation algorithm: space smoothing-based M2M4 (SS-M2M4), which modifies M2M4 with spatial smoothness, a technique that is used in the field of signal reconstruction in graphs. This algorithm can provide a more accurate SNR for the proposed CHMM-based spectrum sensing method.

6.1. Typical SNR Estimator

M2M4 is one of the most widely used blind estimators [14]. M2 and M4 represent the second and the fourth moment of , respectively, where refers to the samples of the received signal. M2 and M4 can be calculated as follows [14]

Then with second-order moments and fourth-order moments, we can estimate SNR as follows,

In practice, the second and fourth moments are usually calculated by their own time averages:

6.2. SS-M2M4 SNR Estimation Algorithm

In CUAVNs, the CUAVs within the same cluster are close to each other, and thus, their large-scale fading and shadow fading are generally similar [40]. Therefore, their SNRs are correlated. However, the M2M4 algorithm estimates these SNRs individually, ignoring the spatial correlation among the SNRs of those neighbouring users. CUAVNs have their own topology, and the SNR estimation problem can be naturally modeled as graph signal processing problems. Therefore, we propose a novel SNR estimation algorithm as shown in Algorithm 1, which considers the smoothness between neighbors [15]. Firstly, the M2M4 algorithm is applied to calculate the SNR of each CUAV, and the estimation result can be represented as . After that, to ensure that the secondary CUAVs that are close to each other are in the same cluster, the max–min distance clustering algorithm described in Section 3.1 is adopted. Based on the clustering result, we consider each cluster as a new graph, and then correct the original SNR to get the final estimated SNR by Equation (42).

The problem of estimating from the original SNR can be modeled as the following optimization problem, where is a high-pass graph filter, and is the regularization parameter. The first term penalizes the error of the estimated graph signal, the second term encourages the smoothness of the estimated SNR. Similar to [15], we set , where is the Laplacian matrix, so that . The smoothness of the estimated graph signal can be characterized by the graph Laplacian quadratic form

It can be written as where is the set of edges. The smaller the function value is, the smoother the SNR difference of the cluster, especially when neighboring CUAVs connected by an edge with a large weight have similar values [15]. We can then represent the optimization problem as

To get the best estimated SNRs, we take the derivation of the formula, and get the optimal solution as

6.3. CHMM-Based Spectrum Sensing with SS-M2M4 SNR Estimation

The estimated SNRs can provide better SNR information for the CHMM-based spectrum sensing procedure, and thus we can obtain more accurate fusion weight in a heterogenous two-stage-fusion stage. Therefore, the CHMM-based spectrum sensing scheme with estimated SNRs achieves a higher detection probability, and the utilization of spectrum sensing can be further enhanced.

The proposed continuous hidden Markov model based spectrum sensing with space smoothing M2M4 SNR estimator can achieve good detection performance in CUAVNs, since it has some advantages over the existing HMM-based spectrum sensing method and the existing SNR estimators. The proposed method can either achieve better performance or be more suitable for UAV applications. The characteristics of these existed methods compared to the proposed method are summarized in Table 2.

Table 2

The characteristics of the existing methods.

Methods	Characteristics	Examples
DHMM-based spectrum sensing	Errors caused by quantization degrade detection performance	Suguna et al. [21], Eltom et al. [22]
CHMM-based spectrum sensing	Centralized spectrum sensing and lack of dynamicity, not suitable for CUAVNs	Halaseh et al. [23], Cheng et al. [24]
Deep learning-based spectrum sensing	Poor interpretability and missing of the sensing term caused by the large delay	Xie et al. [16], Xie et al. [17]
SNR estimator with pre-knowledge	Hard to get pre-knowledge of channel in UAVs applications	Raza et al. [26], Manesh et al. [27]
SNR estimator designed for specific signals	Not generalized for UAVs applications	He et al. [28], Chen et al. [29]
Deep learning-based SNR estimator	Missing of the sensing term caused by the long estimation time	Ngo et al. [31], Yang et al. [32]

7. Evaluation and Numerical Results

In this section, the validity of the approximation of folded normal distribution is firstly presented. Then, the detection performance of the proposed CHMM-based spectrum sensing scheme is evaluated by comparing with the non-CHMM ones. Next, we verify the effectiveness of the proposed SS-M2M4 estimator and further demonstrate the performance of the unified CHMM-based spectrum sensing with the SS-M2M4 estimator. Finally, we further consider mutipath effects in CUAVNs, and verify the effectiveness of the proposed scheme under the Rice channel.

7.1. Approximation of Folded Normal Distribution

The cluster heads adopt cyclostationary detection to sense the primary spectrum. Under the hypotheses , the cyclostationary detection statistic x follows the folded normal distribution, which can be represented as

Figure 5 shows the probability density function of the cyclostationary feature under hypotheses when , . As shown in Figure 5, the folded normal distribution mainly coincides with the normal distribution. When x is less than 0.0025, which is already away from the mean, the two distributions start diverging. In fact, the two terms of folded normal distribution can be seen as two normal distributions with opposite means and the same variance. In spectrum sensing, when the channel is occupied, the mean is away from 0, and it is much larger than the variance. Thus, when , the second term of contributes little to the folded normal distribution. Therefore, in CUAVNs, folded normal distribution can be seen as its first term, that is, a normal distribution in the positive axis.

Figure 5

Approximation of folded distribution.

7.2. Performance of CHMM-Based Spectrum Sensing with Perfect SNR Estimation

In the simulations, the number of secondary CUAVs is set to 20. The CUAVs move according to the random walk mobility model [34], in which the maximum velocity of the CUAVs is 36km/h, and the sensing time is 20 μs. The range of the SNR in our CUAVNs is set as [−15, −3] dB according to [32,33,41]. Locations of secondary nodes lead to different SNRs, we assume that the maximum distance from the transmitter to the secondary UAV is about 1.5 to 2 times the minimum [1]. According to the large-scale fading calculation formula [40], assuming that the fading coefficient is 2, we can obtain 5 dB as a SNR range in our simulation, in other words, the maximum SNR difference of secondary users is within the range of 5 dB. We assume that all secondary CUAVs experience additive white Gaussian noise.

According to [42], the spectrum utilization rate below 3G (The Federal Communications Commission (FCC) of the United States and the European Union (EU) have set 2.4 GHz and 5.8 GHz as the band of civil UAVs. The EU also allocates 433 MHz and 863–870 MHz to UAVs. Similarly, China has set 840.5–845 MHz, 1438–1444 MHz and 2408–2440 MHz as that for UAVs. Compared to 5.8 GHz, more UAVs work on lower bands, below 3 GHz, thus the dilemma of spectrum scarcity is more serious on the below-3G bands) in Berkeley is about 0.3, therefore, we set the initial distribution of the spectrum state distribution as , that is to say, the probability of spectrum presence is 0.3, and the probability of spectrum absence is 0.7. We assume a 1st order Markov chain, and the spectrum state at time t is known, where the distribution is either or . As stated before, the spectrum utilization is around 0.3, the transition probability of absence to presence is thus set as 0.25, and in the same way, the transition probability of presence to presence is set as 0.35. Therefore, the transition matrix is . Next, we use MATLAB to generate a spectrum state sequence with a length of 8000 under the parameters above. According to each single spectrum state (hidden state) and different clustering results, the simulated energy detection statistics and cyclostationary statistics are obtained, respectively. Then corresponding observation values are calculated according to the two-step-fusion method.

Figure 6 and Figure 7 show the received operating curve (ROC) of the CHMM-based sensing method and non-CHMM-based ones under the AWGN channel with 20 CUAVs. In Figure 6, the soft–soft represents the heterogeneous two-stage sensing scheme represented in Section 2.2, in which both the intra-cluster fusion stage and the inter-cluster fusion stage adopt a soft combining rule [43]. It can be observed from Figure 6 that the soft–soft heterogeneous sensing scheme with CHMM predication outperforms the non-predication one (soft–soft). Due to CHMM avoiding the distortion caused by the discretization of DHMM, the proposed CHMM-based sensing scheme can further improve the detection probability compared with the DHMM-based soft–soft scheme, which adopts DHMM to model the sensing procedure. What stands out in Figure 6 is the achieved high detection probability of 0.91 when false alarm probability is around 0.1. A higher with small indicates that the proposed algorithm can offer the secondary CUAVs more opportunities to access the spectrum and maintain tolerable interruption to the primary user.

Figure 6

ROC of CHMM soft–soft scheme (the proposed CHMM-based heterogeneous two-stage fusion sensing scheme), DHMM soft–soft scheme and non-CHMM soft–soft scheme.

Figure 7

ROC of soft–or, or–or scheme and CHMM soft–or, or–or scheme.

In addition, we also simulate the soft–or and or–or schemes [36] to verify the universality of the CHMM model for spectrum sensing in Figure 7, where soft–or means a soft combining rule at the intra-cluster fusion stage and/or a combining rule [43] at the inter-cluster fusion stage. Similarly, or–or means an or combining rule at both the intra-cluster fusion stage and the inter-cluster fusion stage. As shown in Figure 7, in addition to the good detection performance offered by the heterogeneous soft–soft scheme, the proposed CHMM can also achieve obvious improvement when it is implemented into the other two fusion schemes: or–or and soft–or. Then, with the help of a better sensing performance, the spectrum efficiency and throughput can be further improved.

7.3. Performance of CHMM-Based Spectrum Sensing with SS-M2M4 SNR Estimator

In this section, the simulation evaluation of our proposed SS-M2M4 in comparison with the existing M2M4 algorithm is presented. The simulation results with SNR in [−15, −3] are shown in Figure 8. Note that the SNR = −15 dB represents the SNR range from [−15, −10]. Similarly, −3 dB represents the range [−3, 2]. is used to measure the effect of SNR estimation, and the calculation scheme of is given as where represents the estimated SNR of the ith CUAV, represents the actual SNR of the ith CUAV. Figure 8 shows that the proposed scheme offers a good SNR error decrease. It can be seen that Figure 8 also shows that the proposed scheme can improve the performance of SNR estimation up to 3dB (when a is required, the M2M4 estimator needs the actual SNR to be −6 dB, but SS-M2M4 only needs it to be −9 dB). An accurate SNR estimator can further help the CUAV get a better detection performance. Consequently, it is reasonable to expect good detection performance when the CHMM-based spectrum sensing adopts the novel SNR estimation algorithm.

Figure 8

MSE of SS-M2M4 estimator and M2M4 estimator.

Figure 9 shows the CHMM-based spectrum sensing with the SS-M2M4 estimator significantly outperforming the original ones, which either do not use the CHMM model (spectrum sensing + SS-M2M4 SNR estimator) or employ the M2M4 estimator (CHMM spectrum sensing + M2M4 SNR estimator), or neither (spectrum sensing + M2M4 SNR estimator). Specifically, when the false alarm is 0.1, the unified scheme achieves a detection probability of 0.95, while the CHMM-based spectrum sensing with the original SNR estimator (CHMM spectrum sensing + M2M4 SNR estimator) can only achieve around 0.82. Besides, the unified scheme can further enhance the detection performance compared with the DHMM-based spectrum sensing with the proposed SS-M2M4 (DHMM spectrum sensing + SS-M2M4 SNR estimator), and the DHMM-based spectrum sensing with the proposed SNR estimator can also outperform the DHMM-based spectrum sensing with the original SNR estimator (DHMM spectrum sensing + M2M4 SNR estimator). In other words, the proposed SNR estimator has a universality to a different spectrum sensing method and the unified scheme can utilize the spectrum more efficiently and maintain a tiny interference to the primary user.

Figure 9

Detection probability versus false alarm probability.

7.4. Performance of CHMM-Based Spectrum Sensing with SS-M2M4 SNR Estimator under Rice Channel

In the above simulation, we assume the AWGN channel and verify the effectiveness of the proposed scheme under the AWGN channel. In this section, we have further considered the Rice channel when mutipath effects have been introduced in UAV applications [44]. The simulation of the proposed scheme under the Rice channel is shown in Figure 10, according to [44], we set the Rician factor as K = 10. Figure 10 shows that even under the Rice channel, the proposed method (Rice CHMM sensing+SS-M2M4 estimator) can obtain better performance compared with DHMM-based spectrum sensing with the proposed SS-M2M4 SNR estimator (Rice DHMM sensing + SS-M2M4 estimator), CHMM-based spectrum sensing with M2M4 SNR estimator (Rice CHMM sensing+M2M4 estimator), and other methods. Although the detection probability under the Rice channel is lower than that under the AWGN channel, the proposed scheme can also achieve better performance than other schemes under the Rice channel, in other words, our scheme is effective in CUANs.

Figure 10

Detection probability versus false alarm probability under the Rice channel.

8. Conclusions

In this paper, we consider modeling primary user states as a Markov chain, and propose a spectrum sensing scheme based on a continuous hidden Markov model with perfect SNR estimation. We derive the detection probability and false alarm probability of the heterogenous-fusion clustered CUAVNs. Taking the similarity of the SNRs of CUAVs in the same cluster into account, we propose a space smoothing based SNR estimator for sensing in CUAVNs to offer a more accurate SNR to the proposed sensing method. Simulation results show that the unified CHMM-based sensing scheme with the proposed SNR estimator enhances the sensing performance considerably.

The work can be further extended in the following aspects in our future research. First, in the proposed CHMM based spectrum sensing method, the hidden spectrum chain is modeled by first-order Markov chain, and it has not made full use of the existing historical information. In the future research, the high-order Markov chain can be used to model the spectrum sensing process to further improve the accuracy of prediction. Second, in this paper, we considered cognitive UAV networks with a single primary user. Recently, cognitive UAV networks with multi primary users [33] have been proposed to enhance the spectrum utilization. Thus, we can explore our work in multi-PUs CUAVNs in the future.