Literature DB >> 31581585

Gunshot Airborne Surveillance with Rotary Wing UAV-Embedded Microphone Array.

Felipe Gonçalves Serrenho1, José Antonio Apolinário2, António Luiz Lopes Ramos3, Rigel Procópio Fernandes4.   

Abstract

Unmanned aerial vehicles (UAV) are growing in popularity, and recent technological advances are fostering the development of new applications for these devices. This paper discusses the use of aerial drones as a platform for deploying a gunshot surveillance system based on an array of microphones. Notwithstanding the difficulties associated with the inherent additive noise from the rotating propellers, this application brings an important advantage: the possibility of estimating the shooter position solely based on the muzzle blast sound, with the support of a digital map of the terrain. This work focuses on direction-of-arrival (DoA) estimation methods applied to audio signals obtained from a microphone array aboard a flying drone. We investigate preprocessing and different DoA estimation techniques in order to obtain the setup that performs better for the application at hand. We use a combination of simulated and actual gunshot signals recorded using a microphone array mounted on a UAV. One of the key insights resulting from the field recordings is the importance of drone positioning, whereby all gunshots recorded in a region outside a cone open from the gun muzzle presented a hit rate close to 96%. Based on experimental results, we claim that reliable bearing estimates can be achieved using a microphone array mounted on a drone.

Entities:  

Keywords:  direction of arrival (DoA) estimation; gunshot audio surveillance; microphone array; rotary wing drones; shooter localization; unmanned aerial vehicles (UAV)

Year:  2019        PMID: 31581585      PMCID: PMC6806267          DOI: 10.3390/s19194271

Source DB:  PubMed          Journal:  Sensors (Basel)        ISSN: 1424-8220            Impact factor:   3.576


1. Introduction

The interest in automatic sniper localization systems traces back to the early 1990s, pioneered by countries such as the United States of America, Russia, Canada, France, and more recently, Israel, among others. Such surveillance systems for shooter detection and localization can be useful to the police and military forces [1,2]. The shooter detection and localization problem can be approached in different ways, depending on the kind of signatures from a gunshot event, acoustic or electromagnetic, that one decides to process [3]. For instance, cameras can be used to detect the muzzle flash [4], whereas microphone arrays can be used to detect the muzzle blast and the shockwave acoustic signatures. If these two acoustic signatures are detected in the same gunshot event, one can estimate the location of the shooter using a two-step procedure [3]. The successful use of microphone arrays to tackle the direction-of-arrival (DoA) estimation problem even with low signal-to-noise ratio (SNR) can be seen in Reference [5]. In this paper, median filtering is used to enhance the collected acoustic gunshot signals. In Reference [6], an algorithm that optimizes DoA estimation using exhausive search through consistent fundamental loops is introduced. This method, in an attempt to have the best approach for the level of noise of audio signals, is a combination of standard DoA estimation, Exhaustive Search (ES) [7], and consistent fundamental loop [6]. Microphone arrays can be deployed in different platforms, e.g., stand-alone systems mounted on vehicles [8], on light posts in urban areas or on trees in a forest [9]. All these systems are currently subjects of great interest in academia and are recently associated with the internet of things (IoT) industry [10] as well. However, one such system based on a microphone array mounted on an aerial drone brings additional advantages owing to its flexibility to cover wider areas relatively quicker and at a lower cost. It also opens the opportunity for new important applications, such as search-and-rescue missions [11,12] and environmental monitoring [9]. In Reference [11], a microphone array mounted on a drone is used to detect a narrowband signal generated by a whistle, which can be very effective in search-and-rescue missions in areas of difficult access. An application example for environmental monitoring is presented in Reference [9], suggesting the use of an open hardware to be deployed in the forest to record audio signals and using a Secure Digital (SD) card to store the data. These signals vary from bat ultrasounds to gunshot signals. For instance, in the case of detecting gunshot events in protected areas, people responsible for monitoring those areas could be triggered to carry out necessary actions against poaching. Hoshiba et al. presented detailed design and implementation of a quadcopter-embedded microphone array system for outdoor environments [12]. In order to enable new drone applications, the scientific community has developed an interest for new techniques capable of tackling the strong ego-noise presented in audio recordings from unmanned aerial vehicle (UAV)-embedded microphone arrays, especially when the target sound is a whistle or human speech. Methods based on Multiple Signal Classification (MUSIC), known to be very robust against noise, are presented in Reference [13]. The Generalized Eigenvalue Decomposition-MUSIC (GEVD-MUSIC) [14,15] is reported to have high performance even for low-SNR signals. The incremental Generalized Eigenvalue Decomposition-MUSIC (iGEVD-MUSIC) introduced in Reference [16] estimates the noise correlation matrix incrementally to cope with the high non-stationarity of the ego-noise. A supervised approach that uses UAV sensors data and motor rotation speeds to estimate the noise correlation matrix was proposed in Reference [17]. Aiming at reducing computational complexity and errors associated with inaccuracies in noise correlation matrix estimation, Reference [18] proposes the Multiple Signal Classification based on incremental Generalized Singular Value Decomposition-MUSIC (iGSVD-MUSIC) with Correlation Matrix Scaling (CMS). A novel algorithm to sound source location with UAV-embedded microphone arrays based on time-frequency bins was proposed in Reference [19]. This method takes advantage of the fact that ego-noise and target sound (e.g., speech or emergency whistle) mainly consist of harmonic components that usually occupy different time-frequency bins. In Reference [20], the time-frequency technique is associated with a time-frequency spatial filter to enhance the signal of interest. Other interesting researches related to sound processing with drones include the following: a study about the ego-noise of multirotors micro aerial vehicles [21], that also proposes the use of Blind Source Separation (BSS) algorithm to suppress it; the use of ego-noise to measure the relative directions between multirotors in a swarm [22]; and the ability to track moving sound sources [23]. Focusing on gunshot airborne surveillance, the deployment of acoustic sensors in elevated platforms could enable advantages for shooter-position estimation, according to Reference [24]. The use of an aerial drone as a mobile elevated platform was investigated in Reference [25], using only simulations. Different noise levels were synthesized using drone noise recordings in a silent room and real gunshot signals recorded with a high-quality microphone mounted on a tripod in an open field. Also in this work, signal enhancement techniques were employed along with DoA estimation algorithms and target motion analysis was used to estimate the shooter’s position. In Reference [26], the geometry deployment of the microphone array is discussed, taking into account the wind produced by the propellers, the electromagnetic interference, and the scarce space available on the drone. In this paper, we focus on the details of DoA estimation of gunshot signals obtained from a microphone array aboard a flying drone. We used simulations to investigate the performance of preprocessing and DoA estimation techniques, also tuning theirs parameters. The most appropriated methods were evaluated with actual field recordings, given the position and attitude of the drone obtained from its GPS and inertial unit. The rest of this paper is organized as follows. Section 2 starts with a brief overview on gunshot acoustics, followed by a discussion on the techniques used in a UAV-based gunshot surveillance system, namely signal preprocessing, gunshot detection, DoA estimation, and shooter localization. Section 3 describes the hardware used and the shooting site as well as how telemetry data is recovered and used, while Section 4 discusses experimental results from simulations and from actual gunshot signals collected using an array of sensors mounted on a flying drone. The discussion and conclusion are addressed in Section 5.

2. DoA Estimation and Shooter Localization

The acoustic signatures generated by a gunshot event can be divided into three parts, namely the muzzle blast, the shockwave, and sounds related to mechanical actions, which include the trigger and hammer mechanisms, ejection of spent cartridges, and the loading system. The mechanical action-related sounds can be useful in forensics analysis [27,28]. However, they are of no interest in the design of sniper localization systems, since they can only be recorded using sensors placed close to the gun. The muzzle blast is generated by the expansion of gases in the gun barrel and is louder in the direction the barrel is pointing toward [29,30]. It propagates at the speed of sound and lasts typically from 3 to 5 ms [31]. The energy of the muzzle blast depends on the firearm used, and it is almost always audible in a given range, provided that silencers or suppressors are not used [31,32]. A shockwave will be generated for as long as a projectile is travelling faster than the speed of sound and propagates outwards from the bullet trajectory at an angle known as the Mach angle [33]. The shockwave generated by a typical supersonic bullet lasts for approximately 200 µs, and its frequency spectrum has a wider frequency band than that of the muzzle blast, as exemplified in Figure 1. Since the shockwave propagates in a cone shape following the bullet trajectory, it cannot be detected if the bullet is moving away from the position where the sensors are located [34]. This constitutes a problem for shooter localization systems that rely on the detection of both shockwave and muzzle blast signals.
Figure 1

Components of a gunshot signal: shockwave (left) and muzzle blast (right) of a caliber 7.62 mm rifle and the corresponding spectrogram.

The shooter’s localization problem may be divided in four steps, namely preprocessing, gunshot detection and muzzle blast identification, DoA estimation, and shooter-position estimation.

2.1. Preprocessing

Localization of a shooter based on the audio acquired from a drone is especially challenging due to the presence of strong ego-noise, mainly generated by the propellers [35]. This becomes an even greater challenge under long-range shootings, whereby the detection and the direction-of-arrival estimation of the muzzle blast signal is compromised. When signals of interest are approximately stationary, such as tiny voice snippets, whistles, or white noise, methods based on noise correlation matrix estimation, such as Wiener Filtering [36], are used. For impulsive signals, as in the case of gunshot signals, median filtering is an alternative option [5]. In this work, we evaluate the performance of these methods in the task of DoA estimation. During steady parts of a flight, where stationarity can be assumed [36], we may use Wiener filtering to attenuate the influence of the ego-noise. We used in this work the implementation developed by Liu Ming and Pascal Scalart [37,38]. This Wiener filter, referred to as Two-Step Noise Reduction (TSNR), uses the decision-directed approach [39] to track a priori SNR and refines the SNR estimation to avoid reverberation effects. Median filtering was employed in Reference [40] as a technique to separate percussive and harmonic components of a signal. The proposed scheme uses the concept that percussive components can be seen as outliers in the time domain while harmonic sounds can be seen as outliers in the frequency domain. Median filtering, described next, is capable of removing these outliers and of separating these different acoustic signatures to some extent. Given the input , the output is the median value of the window with length centered in . The parameter should be chosen accordingly, in such a way that the expected duration of the artifacts is removed but without significant impact on the signal of interest. The median filtering can be expressed according to the following: if is even, the median is defined as the mean of the two central median values. The use of median filtering to estimate background noise embedded in gunshot signals was introduced in Reference [5], which computes the enhanced signal as . To preserve the muzzle blast’s shape, should not represent less than half of its duration, which is approximately 3 ms [31].

2.2. Gunshot Detection

As previously noted, a gunshot surveillance system must be able to detect an impulsive signal and to identify if it is a muzzle blast component, a shockwave component, or none of them. There is a vast literature available about this matter [41,42,43,44,45,46]. The method in Reference [41] uses a transient detection, introduced in Reference [42], that looks for significant changes in the signal energy. For muzzle blast and shockwave classification, Reference [41] uses two tests: the first one is based on the spectral information of the signal, and the second one uses time difference of arrival between neighboring peaks. A detection scheme based on correlation against a template is proposed in Reference [43], where the authors claim that the method could be implemented by a low power consuming hardware. Correlation against a template is also addressed in Reference [44], where it is compared against classical algorithms usually used in speech processing; their results conclude that correlation matches the performance of those algorithms, especially in noisy environments. In Reference [45], linear predictive coding (LPC) coefficients are combined with template matching to increase the performance of gunshot detection systems, especially regarding false-positive errors. A wavelet-based approach [46] can be used to distinguish three acoustic events: muzzle blast, shockwave, and reflections. Furthermore, according to the authors, this method can classify the caliber based on the muzzle blast or on the shockwave signals. The strong ego-noise of a drone is not white and is highly nonstationary [36]. Furthermore, it is strongly dependent on the drone used and on the positioning of the sensors. These are additional challenges to detection and muzzle blast–shockwave classification. These tasks were carried out manually in this work.

2.3. DoA Estimation Methods

In this section, we first define DoA angles and then present two DoA estimation methods: a data selection least squares method [7] and an angular spectrum-based method named the Multi-channel Blind Source Separation (MBSS) Locate [47]. Figure 2 shows the angles (azimuth and zenith ) that define the DoA. It is noteworthy that the azimuth herein is taken counterclockwise, as in Reference [48]. Thus, the unit vector in the direction of sound wave propagation is given as follows:
Figure 2

Azimuth () and zenith () relative to the center of the array: The x axis is oriented to the front of the drone, and the z axis points upwards.

2.3.1. The Data Selection Least Squares DoA Estimation Algorithm

The first step of the least squares (LS) method is the time-delay estimation (TDE) between the sensor pairs in the array. Next, we use an LS cost function associated with a data-selective algorithm. The TDEs are obtained from the peak of the cross-correlation defined as follows [49]: where E denotes the expectation operator and is the lag between two given sensors, and . In practice, we do not have statistical knowledge of the signals, and Equation (3) is usually approximated by its time average given by the following: where * is the convolution operator. Taking the discrete Fourier transform of and assuming real-valued signals, we can write the cross power spectrum density between and as follows: The cross-correlation can then be computed using the following: Adding a frequency weighting function in Equation (6), we have the generalized cross-correlation (GCC) as follows: where classical cross-correlation corresponds to . A popular weighting scheme employed by the GCC is the phase transform (PHAT) [7,36,49,50,51,52,53], known to have good performance in reverberating scenarios [49]. PHAT also tends to have a sharper peak than classical GCC, increasing the performance of the TDE [50]. The PHAT weighting function is given by the following [53]: Finally, the TDE is obtained as follows: where is the maximum delay possible (in number of samples) between microphone i and j, which occurs when the DoA has the same direction of the vector that connects sensors i and j: where and are the position vectors of sensors i and j, is the speed of sound, and is the sampling frequency. The TDEs using inverse Fourier transform (iFFT) provide delays as integer multiples of the sampling period; this leads to errors that are particularly relevant in small arrays (small time delays between sensors) and with low sampling frequency. To mitigate this source of errors, we can interpolate the GCC, allowing more accurate estimations of the time difference of arrival (TDoA). In this work, we used cubic interpolation [54], calculated at every point in between and , ensuring that all possible values of delay are covered. Figure 3 shows the effect of cubic interpolation over GCC-PHAT in a small array for a signal with kHz.
Figure 3

Effect of interpolation in generalized cross-correlation (GCC)-phase transform (PHAT): Note that is the GCC-PHAT without interpolation.

Figure 4 illustrates in 2-D the delay between microphones i and j. In a 3-D scenario, we write such that the TDE (in samples) is given by the following: where .
Figure 4

Direction of arrival (DoA) calculation in a 2-D scenario.

Based on the estimated delay, as given in Equation (9), and the delay based on the unknown vector , Equation (11), we define the least squares cost function: for all possible pairs, for the case of M microphones. Minimizing the cost function with respect to , we find the following: where , and , are assembled as follows: The solution provided by Equation (13) may not have unit norm, which must be ensured through normalization. Only then could azimuth and zenith be calculated using trigonometric operations, according to Equation (2). Equation (13) provides all three coordinates only when using a spatial array. If a planar array is used, ambiguity occurs and matrix is singular. When all sensors are in a plane (-plane for instance), we must adapt the sensor positions () to suppress the coordinate associated with the perpendicular axis, in our case z. This way is non-singular and Equation (13) provides . As the must be unitary and assuming that the source is located above or below the array, it is possible to estimate the DoA. The strong ego-noise could compromise the TDEs, generating outliers that would adversely affect the DoA estimation. As for the cost function defined in Equation (12), the solution can be obtained without using all available pairs of microphones; it is possible to use a data-selective algorithm to remove outliers. Using the Exhaustive Search algorithm ES(n) [7], we choose the n combination from the set of N pairs of microphones that minimizes the cost function in Equation (12). We need to be cautious when choosing the number of pairs of microphones to be used, parameter “n” in ES(n), once it can generate ill-conditioned matrices [55]. The appropriate choice for “n” can be obtained according to a decision tree as done in Reference [6].

2.3.2. The MBSS Locate

The Multi-channel Blind Source Separation (MBSS) Locate [56] is available as a MATLAB toolbox. It estimates the direction of arrival of multiple sources from audio signals collected by an acoustic sensor array. This software implements multichannel versions of four state-of-the-art and three proposed SNR-based local angular spectra methods for audio signals [47]. The state-of-the-art local angular spectra methods are GCC-PHAT [49] and its version with a nonlinear function GCC-NONLIN [57], Multiple Signal Classification (MUSIC) [13], and Cumulative State Coherence Transform (cSCT) [58]. These techniques, except the cSCT method, rely on the assumption that one source is predominant in each time-frequency bin. The cSCT method assumes that there are at most two predominant sources. The SNR-based local angular spectra tackles the multisource TDoA estimation problem. The main idea is to use the SNR as an unbounded measure to estimate if the information of a time-frequency bin results from a single source. Blandin et al. [47] proposed three methods to estimate the SNR using two microphones and the following techniques: Minimum Variance Distortionless Response (MVDR) [59], Diffuse Noise Model (DNM) [60], and Minimum Variance Distortionless Response Weighted (MVDRW). The MBSS full version enables the user to simulate the recording scenario, e.g., room dimensions, walls absorption coefficient, and number of microphones [56]. Nevertheless, we summarize in the following only the core of the angular-spectra based method. For detailed information about its functionalities and implementation, one should refer to the user guide provided with the software. We describe the use of the MBSS algorithm in three main steps. The first step is to define the possible angles of azimuth and zenith and to assemble the search grid. The program uses elevation instead of zenith, but it can be easily converted: zenith = − elevation. Based on the grid, the set of possible delays for each pair of microphones is computed, and then, it is resampled to limit the quantity to points in which angular spectrum is calculated. The software offers some options to compute angular spectra. For this work, we used the GCC-PHAT local angular spectra defined as follows [47]: where is the real operator, l is the index of the time frame, f is the center frequency of the FFT bin, and t is the delay in seconds. In the second step, the contents of all selected frequency bins are summed up. A linear interpolation is used in to the compute angular spectrum approximation in all possible t in each pair of microphones. This value is used to calculate the angular spectrum directly, depending on the direction of arrival, . Then, angular spectrum of all pairs are summed, generating . For multiple time frames, there are two strategies: sum all time frames or use the maximum overall time frames. The last one is recommended when the signal of interest is only active in a few frames [47]. As the gunshot signals are impulsive, the maximum approach was used and the angular spectrum is then given by the following: The last step of the MBSS algorithm is a grid search to find the global maximum in the case of only one source or the local maxima when there are multiple sources. If there is only one single source, the DoA angles are obtained from the following:

2.4. Position Estimation

There are a number of ways to estimate the shooter localization. A simple approach uses DoA estimations of muzzle blast from different arrays according, for instance, to the total least squares (TLS) [61] algorithm. Since this method does not use the shockwave component, it can estimate position even of small caliber weapons of which the projectiles do not reach supersonic speed. Another advantage of this method is that, with a sufficient number of arrays, it could be combined with a data-selective algorithm, such as Exhaustive Search, seen in Section 2.3, to remove outliers expected to happen when some arrays do not have a clean sight to the firearm or are heavily affected by multipath [31,32]. On the other hand, in order to use the TLS approach, the system would be more complex and expensive to be deployed, since it requires more than one drone and they need to communicate with the node responsible for the calculation of the shooter’s position with the information of all platforms. A second approach is to combine shockwave and muzzle blast DoA estimations to compute the probable shooter location [41,62]. As this method uses shockwave components, it is only applicable in the case of supersonic projectiles and when the array is inside the shockwave field of view. Moreover, this method assumes (at least its simplest version) that the projectile has a constant speed, which tends to generate results that overestimate the distance when the shooter is more than 100 m from the array; adaptations are then required to overcome this limitation as stated in Reference [63]. A third approach presented in Reference [24] combines muzzle blast DoA estimation from an elevated array with a digital map containing topographic information to estimate the shooter position. The main concern of this method is to obtain the appropriate digital model of the terrain. As in the TLS approach, this method can estimate the position of subsonic firearms. This approach would be appropriated for our scenario, but focusing on the DoA estimation, we have not carried out a position estimation evaluation in this work.

3. System Setup and Signal Acquisition

In this section, we describe the hardware, the drone, and the microphone array used in field recordings. We also provide some information about the shooting site and the environmental conditions the experiment was performed under. We also provide details regarding the data acquisition process, including audio recording and drone flight log data.

3.1. UAV and Avionics

We used a DJI Phantom 4. It weights 1.38 kg (battery and propellers included but without the extra hardware used for recording the audio signals) and has a 35 cm diagonal, also featuring a 4K camera, and support for two satellite positioning systems (GPS nad GLONASS). According to the manufacturer [64], the UAV, without any external hardware, is able to resist to wind gusts up to 36 km/h. The microphone array was mounted in 41 cm metal rods, aligned with the propeller’s arms. The size of the rods was engineered to keep the microphones away from the propellers to reduce the influence of noise caused by air displacement generated by the rotating blades. The four microphones were placed in the same height in a planar structure to avoid interference with the drone’s maneuverability, especially during take off and landing. The planar coordinates for the microphones are given in Table 1, assuming the origin of the coordinate system in the center of the UAV.
Table 1

Planar array coordinates.

Microphonex (cm)y (cm)
126.5 25.5
226.527.0
3 25.0 26.0
4 25.0 25.5
The gimbal and the camera were removed, allowing the recorder to be placed under the drone (see Figure 5), aligning it with the center of mass of the multirotor, and minimizing the impact on the flight capabilities of the UAV. Care was taken in order not to cover the ultrasonic sensors, located on the underside of the drone’s hull; this would severely affect flight safety and its landing ability.
Figure 5

Drone used in the experiments: (a) landed; (b) during flight.

3.2. Environmental Conditions and Shooting Site

The gunshot signals were collected in a shooting site located at the Brazilian Army Evaluation Center (CAEx) on a cloudy day with no strong wind and with a temperature of 24 C. Figure 6 shows a satellite image of the shooting site. The drone’s flight zone was restricted to the blue rectangle of area 30 × 120 square meters to prevent it from flying over sensitive regions and to ensure a clear line of sight to the shooter.
Figure 6

Shooting site: In red (marker) is shooter location, and in blue is the allowed flight zone of the drone. Adapted from Google Maps [65].

3.3. Data Acquisition: Audio and Drone Position and Attitude

The four microphones were connected to a four-channel recorder, TASCAM DR-40 [66], which is convenient given its relatively reduced dimensions and light weight of 0.213 kg without batteries. The TASCAM DR-40 recorder comes with two connectors for external microphones and two built-in microphones, which were rearranged to a single set with four external channels to accommodate four small electret microphones. The recordings were carried out using a sampling frequency of 44.1 kHz and encoded using 24 bits per sample. The drone flight log data was recorded in a file and recovered using [67]. The log data provides the following information: time (in ms), GPS coordinates (latitude and longitude), altitude, and attitude data (angles yaw, roll, and pitch), as illustrated in Figure 7.
Figure 7

Attitude angles as measured by DJI Phantom 4, adapted from [68].

The digital audio recorder and the drone were initialized manually and simultaneously for each flight to synchronize the data about the position and the attitude of the drone with the recorded gunshot signals. As the drone was hovering when the shots were fired, the mismatch due to the manual process is negligible. Furthermore, there was no considerable drift caused by two different clocks, since the battery capacity limits the duration of each flight to a maximum of 18 minutes.

3.4. Axis Rotation

The DoA is calculated with respect to the drone’s coordinates of which the axes are not necessarily aligned with the geographic axes. Therefore, after calculating the DoA with respect to the drone’s coordinates, we must rotate the DoA vector in order to match the orientation of the geographic axes. The rotation can be applied by a series of matrix multiplications [69], using the attitude data and the magnetic declination of the location. Considering the axes system shown in Figure 2, the matrix that computes a rotation over axis z (yaw-) is given by the following: The rotation matrix over axis y (pitch-) is given by the following: Also, the matrix that performs the rotation over axis x (roll-) is given by the following: Therefore, the rotated DoA vector in the geographic coordinate system is expressed as follows: Please note that matrix multiplication is not commutative, and therefore, the sequence roll, pitch, and yaw must be respected. Furthermore, the coordinates systems in Figure 2 and Figure 7 are not the same: axes y and z point in opposite directions; it is necessary to reverse the rotation directions of pitch and yaw angles given by DJI Phantom 4. We must also take into account magnetic declination when rotating over axis z Equation (18), or the DoA vector will be aligned with magnetic north instead of geographic north.

4. Experimental Results

4.1. Simulated Signals

In this work, we used simulated muzzle blast gunshot signals with different noise levels to tune parameters and to evaluate the performance of DoA estimation methods. In order to evaluate the quality of a DoA estimation, we used the angle between the estimated and the actual DoA, herein named angular error and defined as follows: where is the correct DoA vector and is the estimated one. Angular error can vary from 0, when there is no error in DoA estimation, up to 180, when DoA estimation points towards the opposite direction of actual DoA. This metric allows us to compare objectively two different estimations while avoiding distortions in azimuth error when zenith is close to 0 or 180. We used three performance measures based on angular error to evaluate the DoA estimation methods: mean, standard deviation, and percentage of estimations with angular error less than 3. An error of 3 is expected to cause an error of approximately 6.28 m at the 120 m range. The simulation of the muzzle blast signal used 7 real gunshot recordings from a rifle Fz 7.62 M964 (FAL) manufactured by Indústria de Material Bélico do Brasil (IMBEL) [70]. Signals were collected with a high-quality microphone in an open and quiet environment, avoiding distortions such as additive noise and multipath propagation effect. These clean gunshot signals were clipped to be 10 ms in length. The selected muzzle blast was considered as the signal of a virtual microphone in the center of the array. Then, we inserted fractional delays to generate each one of the microphone’s target signal, simulating the spatial position of the sound source with respect to the array. Noise was simulated based on eighteen recordings made during flights of the drone with the setup described in Section 3.1. During these recordings, the drone was hovering at different altitude levels, ranging from 10 m to 50 m. At each iteration of the simulation, a random muzzle blast signal and a random noise file were selected. Next, the noise file was clipped at a random point with the size of the desired window. As the noise may have different magnitudes for each microphone, we define SNR as the mean SNR across all the sensors: where M is number of sensors in the array, is the variance of the noise in the ith sensor, and is the variance of the muzzle blast component in the ith sensor defined from a 10-ms window. We divided the results of simulations in two groups: LS method and MBSS, each one having its own parameters to be optimized. In both cases, we studied the effectiveness of preprocessing techniques. In this experiment, we ran 3000 iterations for each SNR value. In each iteration, the DoA was drawn according to a uniform distribution over a semisphere (as already mentioned, we consider that the shooter is in a lower position when compared to the drone). For the LS method, simulations aimed at the best values of window size (from 20 ms up to 50 ms) and n in ES(n). As the array is composed of 4 sensors, pairs of microphones are available, so we tested from to . Analyzing Table 2, we note that the best estimation was usually obtained using the smallest window. This was expected, since the muzzle blast signal lasts 10 ms and a smaller window would contain less noise without losing information about the muzzle blast signal. As stated in Reference [6], an optimal n depends on the SNR: when there is less noise, we should consider more pairs; conversely, when the SNR value gets lower, more pairs have their TDEs compromised and should be discarded. As for the preprocessing techniques, we note that the median filter improves the quality of DoA estimation. However, the Wiener filter implementation used herein did not fit well to the application at hand when combined with the GCC-PHAT. An in-depth analysis using the complete results of the LS simulation in Table A1, would indicate that the median filter has better performance among all estimates with angular errors less than 3.
Table 2

Least squares (LS) method simulation: The best parameters per signal-to-noise ratio (SNR).

Without PreprocessingMedian FilterWiener Filter
SNRdBn/WindowSize (ms)Error < 3° (%)n/WindowSize (ms)Error < 3° (%)n/WindowSize (ms)Error < 3° (%)
104/3599.60006/5099.80004/5088.4333
54/2085.93334/2098.90004/5039.6333
24/2056.96674/2096.76674/5013.7667
04/3535.36673/2089.80004/506.8000
−23/2019.20003/2074.03334/502.4667
−33/3513.06673/2059.53333/501.4333
Table A1

LS method simulation: complete results.

Without PreprocessingMedian FilterWiener Filter
SNRdBn/WindowSize (ms)MeanError (°)StandardDeviation (°)Error < 3°(%)MeanError (°)StandardDeviation (°)Error < 3°(%)MeanError (°)StandardDeviation (°)Error < 3°(%)
103/200.51053.926999.10000.32590.511099.300019.417429.260063.9667
3/350.41132.956999.33330.31660.489799.23337.516819.834885.4667
3/500.54984.317898.80000.31370.472099.33336.785918.875986.3667
4/200.3577 2.5648 99.43330.29560.456699.466714.278425.621067.2667
4/35 0.3456 2.5860 99.6000 0.29370.459899.33335.711617.166486.8000
4/500.44923.785999.53330.29830.465299.4333 4.9097 15.8202 88.4333
5/200.72664.265597.33330.25700.406599.600017.362723.831449.3333
5/350.64154.046597.90000.25880.415099.60007.296217.081277.1333
5/500.63434.161198.03330.25990.420699.60006.252815.960280.3000
6/203.86319.262379.70000.21550.411399.766720.903021.000631.3000
6/353.63648.780980.56670.21110.3641 99.8000 11.407617.044255.8333
6/503.61978.742380.3000 0.2081 0.3555 99.8000 10.156216.094659.5667
53/208.943821.772183.03330.45091.872498.766747.879427.955013.4000
3/359.039921.919382.93330.39800.605798.766735.885731.706633.7000
3/509.277221.974282.30000.47501.964498.133333.390931.634337.4333
4/20 6.3045 18.2118 85.9333 0.3558 0.5525 98.9000 43.640928.984815.2333
4/356.753218.976285.26670.36340.547498.866732.897431.285334.8000
4/506.699218.573784.96670.39601.015098.5000 30.4495 31.4588 39.6333
5/209.048618.064568.83330.35761.090198.886743.588726.25807.9333
5/358.970718.534569.1333 0.3306 0.7038 98.9000 33.116028.515123.4333
5/508.743317.727569.23330.37171.404398.833331.287928.812726.4667
6/2016.335517.795634.00000.96644.518196.166742.2534 22.5981 3.1000
6/3516.167217.680433.66670.66252.985896.966733.727923.519911.7333
6/5015.8325 17.4014 34.90000.93304.563196.566732.158423.762613.8333
23/2025.000231.269054.23331.20666.501595.966754.870023.42662.7333
3/3525.107131.275753.86671.20676.051696.033349.762927.401611.4333
3/5025.480731.200152.86671.38707.582495.366748.907627.998913.2667
4/20 20.1555 28.8475 56.9667 0.8764 4.7645 96.7667 52.921723.93412.6333
4/3520.387329.089256.73330.88034.949496.733348.323828.155811.7333
4/5020.628329.048656.10000.9061 4.2183 96.233347.345928.5305 13.7667
5/2022.829625.808135.76671.43676.068693.666752.356922.89730.9667
5/3522.719226.222537.60001.49926.172393.466747.806226.47876.6000
5/5022.369525.716336.50001.87497.266892.200046.641727.08578.6667
6/2027.980820.998512.40004.695211.296880.100050.6875 21.5335 0.3667
6/3527.522821.090613.30004.499110.749879.800046.089423.39742.3000
6/5027.2111 20.6884 13.06675.606112.574977.4667 44.6994 23.93383.7000
03/2036.537231.977033.40003.869713.9666 89.8000 56.011422.11680.7667
3/3536.530631.865133.23333.728413.170088.233354.150724.30324.6333
3/5036.613631.920333.60005.754017.539884.966752.731125.05676.1333
4/20 31.5958 31.228235.30003.649713.906189.200054.820922.33060.9000
4/3532.065331.4251 35.3667 3.5862 12.6191 87.966753.190424.87875.0667
4/5032.148131.525535.23335.339616.282683.933351.826225.7706 6.8000
5/2032.775327.804320.33335.826914.514377.233354.492121.74180.1333
5/3532.984327.826418.70006.162614.373375.200052.864323.98312.4000
5/5033.852128.158719.33337.485916.404972.533351.471424.81033.5667
6/2034.919522.26856.100011.673917.723556.800053.4600 21.1648 0.2333
6/3535.2111 22.2109 5.666712.720218.104352.566750.696422.52400.7000
6/5035.579022.40755.900013.946819.189850.1667 49.6944 22.58820.9333
−23/2045.141330.2568 19.2000 12.3675 26.0754 74.0333 56.625821.74860.1333
3/3545.909129.635617.733314.783028.475868.833356.185722.80851.6667
3/5045.660429.825418.366717.272130.477263.366755.817523.01732.2333
4/20 42.2719 30.578419.166712.494025.059267.933355.880421.43360.1000
4/3542.754030.276217.766714.967127.477963.300055.601723.05521.9667
4/5042.711830.157117.600017.038328.842258.133355.097223.3792 2.4667
5/2042.486027.68018.866715.7599 23.6227 51.566755.541921.44170.0667
5/3542.646127.30378.366717.856425.171046.100054.961622.80000.9000
5/5043.137127.22148.200019.790725.976641.566754.740723.07731.0667
6/2042.316523.03142.533322.901323.637230.600054.8717 21.0324 0.1333
6/3542.6043 23.0215 2.133324.521323.938826.733353.422122.05510.2333
6/5042.674723.05242.433326.486924.345824.3000 53.2176 22.32570.3000
−33/2048.950328.009212.466720.076732.1463 59.5333 56.573321.76650.1667
3/3549.283728.3516 13.0667 23.463234.291555.1000 56.6267 22.11721.0000
3/5049.329028.312612.600026.506835.684349.100056.196622.5202 1.4333
4/2047.145128.578111.9000 19.7724 31.002553.933356.292821.44320.1333
4/3546.788028.819812.600022.969032.641749.700056.438622.32541.1000
4/5047.400528.484211.500026.241633.831243.366755.905122.74391.3333
5/2046.726926.27935.200022.295627.351138.300056.007321.21510.1333
5/3546.393626.57455.766725.108429.195434.566756.149722.06780.5667
5/5046.990426.56225.700028.220230.065930.066755.494922.43930.6000
6/2045.9073 22.8674 1.133328.528224.719021.066755.4752 21.0532 0.1333
6/35 45.7383 22.99031.300031.070225.838118.233354.811222.02720.2333
6/5046.316423.21091.166732.9346 25.6870 15.766754.283021.96270.2667
In our simulations, we defined two basic MBSS parameters: grid resolution, which was set to 1, and alpha resolution, which was set to 0.5. The first one is the minimum increment considered in DoA angles, while the second one is related to the resample of possible delays for each pair of microphones, as mentioned in Section 2.3.2. These parameters do not have a considerable influence on performance with low SNR. Assuming that a muzzle blast would come from below the drone, the search boundaries for azimuth and zenith were set to 0 to 359 and 90 to 180, respectively. We explored the most suitable values for window and frame sizes; the former varied from 25 ms up to 50 ms, and the latter varied from 10 ms up to 20 ms. A summary of the MBSS simulation results containing the best parameters per SNR in relation to the rate of estimations with angular error less than 3 are shown in Table 3. We noted that frame-based processing, together with the overall maximum strategy, led to the best performance with a 50-ms window size and a frame size of 12 ms or greater. We also notice that the MBSS method does not work well with the preprocessing techniques previously mentioned. Nevertheless, MBSS proved to be more robust to ego-noise, achieving high hit rates even for SNRs as low as dB. The complete results of MBSS simulation can be seen in Table A2.
Table 3

Multi-channel Blind Source Separation (MBSS) Simulation: The best parameters per SNR.

Without PreprocessingMedian FilterWiener Filter
SNRdBWindow Size/Frame Size (ms)Error < 3° [%]Window Size/Frame Size (ms)Error < 3° [%]Window Size/Frame Size (ms)Error < 3° [%]
1050/1250/1550/2099.400050/1250/1550/2098.733335/1292.9667
550/1250/1550/2099.166750/1250/1550/2098.666735/1263.3667
250/1250/1550/2098.700050/1250/1550/2098.300035/1229.8000
050/1250/1550/2097.666750/1250/1550/2096.566735/1215.4000
−250/1250/1550/2096.733350/1250/1550/2094.666735/127.5333
−350/1250/1550/2095.933350/1250/1550/2092.366735/125.8333
−550/1250/1550/2090.733350/1250/1550/2078.933335/123.2667
−850/1250/1550/2061.900050/1250/1550/2040.200035/122.1667
Table A2

MBSS simulation: complete results.

Without PreprocessingMedian FilterWiener Filter
SNRdBWindow Size /Frame Size (ms)MeanError (°)StandardDeviation (°)Error < 3°(%)MeanError (°)StandardDeviation (°)Error < 3°(%)MeanError (°)StandardDeviation (°)Error < 3°(%)
1025/ 100.45310.446899.30000.52750.602998.43333.176710.744790.8000
35/100.45010.432899.30000.52200.589698.53338.946120.377180.2000
35/120.44730.425999.36670.50940.556398.6667 2.2018 8.0205 92.9667
50/100.45240.433899.33330.53540.617298.233311.453523.474177.0000
50/12 0.4415 0.4084 99.4000 0.4956 0.5301 98.7333 3.378111.663191.3000
50/15 0.4415 0.4084 99.4000 0.4956 0.5301 98.7333 3.378111.663191.3000
50/20 0.4415 0.4084 99.4000 0.4956 0.5301 98.7333 3.378111.663191.3000
525/ 100.51080.539298.93330.60130.711197.733320.935028.610755.7333
35/100.50690.539398.86670.60150.701997.866731.975331.945939.9667
35/120.48650.493499.06670.55740.624098.2333 16.8103 26.6503 63.3667
50/100.49850.510099.03330.59430.690797.766736.238931.732733.5667
50/12 0.4608 0.4506 99.1667 0.5167 0.5624 98.6667 23.515429.745552.9000
50/15 0.4608 0.4506 99.1667 0.5167 0.5624 98.6667 23.515429.745552.9000
50/20 0.4608 0.4506 99.1667 0.5167 0.5624 98.6667 23.515429.745552.9000
225/ 100.65871.005197.10000.74380.968996.100040.442131.561723.8333
35/100.67602.358897.26670.84663.270296.133346.939630.938416.6333
35/120.58970.724697.96670.68310.832897.0333 36.6388 32.2888 29.8000
50/100.65111.588597.56670.84263.249996.600050.0315 28.4052 11.7667
50/12 0.5097 0.5408 98.7000 0.5769 0.6563 98.3000 42.387731.830922.3000
50/15 0.5097 0.5408 98.7000 0.5769 0.6563 98.3000 42.387731.830922.3000
50/20 0.5097 0.5408 98.7000 0.5769 0.6563 98.3000 42.387731.830922.3000
025/ 100.81221.954495.90001.26825.415893.733348.312329.813812.0667
35/101.18275.833795.40001.63288.018593.466752.349227.86957.3000
35/120.70210.846896.50000.8722 1.5264 95.1667 45.6052 30.6252 15.4000
50/100.85042.512695.76671.17904.365093.766753.1432 26.8928 6.5000
50/12 0.5871 0.7072 97.6667 0.8010 3.4158 96.5667 48.869230.105812.2333
50/15 0.5871 0.7072 97.6667 0.8010 3.4158 96.5667 48.869230.105812.2333
50/20 0.5871 0.7072 97.6667 0.8010 3.4158 96.5667 48.869230.105812.2333
−225/ 101.90499.178192.10003.630914.108288.333353.383427.62595.0333
35/102.223010.123491.96674.637016.570186.666755.502026.48143.5000
35/120.95772.858694.53331.74327.363591.4000 51.7095 29.1208 7.5333
50/101.86689.285893.40003.459214.069289.300055.2182 25.2605 2.9333
50/12 0.6788 0.8690 96.7333 1.2045 5.6296 94.6667 53.476927.67945.8000
50/15 0.6788 0.8690 96.7333 1.2045 5.6296 94.6667 53.476927.67945.8000
50/20 0.6788 0.8690 96.7333 1.2045 5.6296 94.6667 53.476927.67945.8000
−325/ 102.785712.165690.66676.766420.368182.966754.279427.45134.6667
35/103.486313.878489.66678.375522.422880.200056.070025.90852.4000
35/121.33765.684392.86673.508514.016287.4667 52.8001 28.5094 5.8333
50/102.634411.697291.80006.283620.045384.733356.5674 24.8849 2.2333
50/12 0.9751 4.4240 95.9333 2.0194 9.4220 92.3667 54.900027.07544.2667
50/15 0.9751 4.4240 95.9333 2.0194 9.4220 92.3667 54.900027.07544.2667
50/20 0.9751 4.4240 95.9333 2.0194 9.4220 92.3667 54.900027.07544.2667
−525/ 108.999523.779178.300017.898532.596962.900055.568626.82712.1333
35/1012.111127.974274.633322.277935.484159.033357.170326.12661.1000
35/124.499815.666884.300012.664028.445369.9000 54.9973 27.9755 3.2667
50/108.530223.374580.300019.030733.828764.866756.8281 24.4191 0.9000
50/12 3.0576 13.0383 90.7333 9.1981 24.1209 78.9333 56.341927.03531.8667
50/15 3.0576 13.0383 90.7333 9.1981 24.1209 78.9333 56.341927.03531.8667
50/20 3.0576 13.0383 90.7333 9.1981 24.1209 78.9333 56.341927.03531.8667
−825/ 1033.531139.817741.233344.303540.991726.633357.201226.89901.8000
35/1036.334539.972438.066746.846641.095626.000057.518225.59220.8667
35/1221.741634.773553.666735.2805 40.2148 35.5667 56.5289 27.8457 2.1667
50/1034.008239.606443.400046.591240.833725.933357.7520 24.7207 0.6667
50/12 18.8424 33.5860 61.9000 34.4365 40.5661 40.2000 57.258026.39921.7000
50/15 18.8424 33.5860 61.9000 34.4365 40.5661 40.2000 57.258026.39921.7000
50/20 18.8424 33.5860 61.9000 34.4365 40.5661 40.2000 57.258026.39921.7000
Based on the simulation results, we chose 2 schemes to process the real gunshots: MBSS with window size of 50 ms and frame size of 15 ms and LS-method using n = 4 and frame size of 20 ms preprocessed with median filtering.

4.2. Field Recordings

The recordings were carried out in 5 sets. In the first one, 3 shoots were recorded only to make sure that the system was fully operational and were not used to evaluate its performance. The next four sets contain, respectively, 50, 50, 60, and 87 gunshot recordings. Summing up, we have a total of 250 gunshots, all from a Carbine IMBEL 7.62 IA2 [70]. In each series, the drone’s flight height varied from 8.8 m up to 60.5 m. The upper limit of flight height was set to ensure safety since the additional payload in the drone compromises its ability to withstand wind gusts. In some recordings, both muzzle blast and shockwave components are present, while in cases where the drone was not positioned in the propagation path of the shockwave, only the muzzle blast is present. In this work, we address DoA estimation of the muzzle blast only. In order to avoid issues related to automatic detection, the system recorded continuously for the duration of the flight, and signals were clipped, manually preserving the muzzle blast only. These two acoustic signatures overlapped in a few recordings. When analysing the results, we found out that azimuth estimations were biased. The bias was similar in the third, fourth, and fifth sets but clearly different for the second run. This, combined with the fact that the UAV required a calibration of its magnetic sensor between the second and the third runs, indicates that this bias can be credited to electromagnetic interference in electronic compassed caused by other circuits aboard. As this bias was spotted only when processing the signals in the laboratory, the value of the compensation had to be estimated directly from the gunshots. To estimate the bias, we computed the mean azimuth error, but in order to mitigate the deleterious effects of possible outliers, we used only DoA estimations with zenith errors less than 3. Finally, we obtained bias correction values of −8.6743 for the second set and −16.7746 for the third, fourth, and fifth sets. The experiments were designed to evaluate the performed of the algorithms under different controlled values of SNR for the gunshot signals. However, other important measurements from GPS and attitude sensors are assumed to be inherently noisy. Table 4 presents the results obtained for the 247 muzzle blast gunshot signals under test. Although for the simulated signals we used as the hit rate the percentage of shots with angular errors lower than 3, for the real gunshot signals, we increased this threshold to 10. The results in Table 4 represent an average of the different recording conditions, depending on the position of the drone as shall be seen in the following.
Table 4

Experimental data of muzzle blast DoA estimation.

Mean Error (°)Standard Deviaton (°)Error < 10° (%)
LS + MF8.38237.221570.4453
MBSS9.645112.211372.8745
Figure 8 illustrates the relationship between the position of the drone and the DoA estimation error. Notice that, as the distances between drone and shooter are not substantially large, 120 m at most, the error observed in this experiment is not strongly related to the distance but is rather correlated to relative position: when the drone is within a cone in front of the weapon, the results are poorer. We analyzed the recordings from positions within this cone and observed distorted signals in most of them. These positions are in the field of view of the shockwave but also in the direction of and in a small distance from the gun barrel. This suggests that the causes of the distortions are twofold: overlap of shockwave and muzzle blast components and a great signal intensity saturating the sensor. In an attempt to measure the system performance in better positioning, we took all gunshots recorded in a region outside a 35 cone open from the weapon muzzle and the error dropped considerably. The hit rate increased to 92.86% for the MBSS technique and to 95.54% for the LS + MF method instead of the former 72.87% and 70.45%, respectively.
Figure 8

Positions of the drone and respective DoA angular errors: Note that the greater errors (warmer colors) correspond to a region in front of the muzzle. (a) Results of the Least Squares method with median filtering. (b) Results of the MBSS method without preprocessing.

5. Discussion and Conclusions

In this work, we analyze the problem of determining the position of a shooter based on gunshot signals acquired using a microphone array mounted on a multirotor UAV. We have conducted a comprehensive literature review on essential topics characterizing the state-of-the-art for this kind of application. We narrow down the focus on the main task, which is to determine the direction of arrival for the muzzle blast and to evaluate the performance of two well-established DoA estimation techniques as well as two important preprocessing methods. We carry out extensive simulations to evaluate the performance of DoA algorithms and to tune their parameters before finally testing the methods with actual gunshot dates recorded in a firing range. Based on our experimental results, we claim that an aerial microphone array mounted on a drone can be used to obtain good estimates of gunshot direction of arrival using different techniques. The experiments also highlight the fact that the accuracy of the estimates are sensitive to the drone position relative to the shooter and emphasize that better results can be achieved with a system that can fly at higher altitudes, in which case it would be possible to estimate the position of the shooter as well. Nevertheless, issues like detection, classification, and noise cancellation algorithms require further investigation, testing, and validation to achieve a fully functional, reliable, and autonomous system.
  3 in total

1.  Variations in recorded acoustic gunshot waveforms generated by small firearms.

Authors:  Steven D Beck; Hirotaka Nakasone; Kenneth W Marr
Journal:  J Acoust Soc Am       Date:  2011-04       Impact factor: 1.840

2.  Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments.

Authors:  Kotaro Hoshiba; Kai Washizaki; Mizuho Wakabayashi; Takahiro Ishiki; Makoto Kumon; Yoshiaki Bando; Daniel Gabriel; Kazuhiro Nakadai; Hiroshi G Okuno
Journal:  Sensors (Basel)       Date:  2017-11-03       Impact factor: 3.576

3.  Deploying Acoustic Detection Algorithms on Low-Cost, Open-Source Acoustic Sensors for Environmental Monitoring.

Authors:  Peter Prince; Andrew Hill; Evelyn Piña Covarrubias; Patrick Doncaster; Jake L Snaddon; Alex Rogers
Journal:  Sensors (Basel)       Date:  2019-01-29       Impact factor: 3.576

  3 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.