A Systematic Review on Particle Swarm Optimization Towards Target Search in The Swarm Robotics Domain.

Swarm Intelligence (SI) is one of the research fields that has continuously attracted researcher attention in these last two decades. The flexibility and a well-known decentralized collective behavior of its algorithm make SI a suitable candidate to be implemented in the swarm robotics domain for real-world optimization problems such as target search tasks. Since the introduction of Particle Swarm Optimization (PSO) as a representation of the SI algorithm, it has been widely accepted and utilized especially in local and global search strategies. Because of its simplicity, effectiveness, and low computational cost, PSO has retained popularity notably in the swarm robotics domain, and many improvements have been proposed. Target search problems are one of the areas that have been continuously solved by PSO. This article set out to analyze and give the inside view of the existing literature on PSO strategies towards target search problems. Based on the procedure of PRISMA Statement review method, a systematic review identified 51 related research studies. After further analysis of these total 51 selected articles and consideration on the PSO components, target search components, and research field components, resulting in nine main elements related to the discussed topic. The elements are PSO variant, application field, PSO inertial weight function, PSO efficiency improvement, PSO termination criteria, target available, target mobility status, experiment framework, and environment complexity. Several recommendations, opinions, and perfectives on the discussed topic are presented. Finally, recommendations for future research in this domain are represented to support future developments.

Introduction

One of the natural mysterious and wonders are how the natural swarm such as birds, bees, ants, and fish adapt and evolved in the challenging and demanding environment just for the species continuity and surviving purpose. The key success of these natural swarms to survive is the ability and intelligence to complete the task that a complex individual such human can do without the need of centralized control while having high robustness and flexibility. This type of intelligence is known as Swarm Intelligence (SI). Since it emerging as one of the Artificial Intelligence (AI) sub-domain in the 1980s, SI has attracted researcher community attention in many disciplines including robotic, engineering, economics, medical, etc. The SI term has first been introduced by Beni and Wang [1] in the context of the application of cellular robotic systems and since then SI based algorithms have growing popularity taking advantage of being flexible and versatile that its can offered. SI is well known for its decentralized collective behavior based on self-organized systems. Examples of complex collective behavior can be observed in aggregation [2], dispersion [3], stigmergy [4], and pattern formation [5] behaviors. SI mechanism works as a population-based system that includes a large number of agents which interact locally with one another and with their environment [6]. The SI algorithm has grown popularity and acknowledgment in the optimization domain especially in NP-hard problems where finding a global optimum becomes almost impossible in a real-time manner. The potential solutions tend to be infinite and finding a workable solution within time limitations is the priority to accomplish. This is where SI can be utilized for solving nonlinear design problems with real-world applications covering trans-disciplinary areas including sciences, engineering, and industries, from computational intelligence and data mining to optimization. Particle swarm optimization (PSO) [7], ant colony optimization (ACO) [8], artificial bee colony optimization (ABC) [9], bee algorithm (BA) [10], bacterial foraging algorithm (BFO) [11], firefly algorithm (FA) [12] and glowworm swarm optimization (GSO) [13] are the main representative of SI optimization strategies. From these representative lists, PSO has been the most researched SI optimization strategy and attract the most research community attentions between 2005 to 2012 [14].

The optimization term can be referred to as a branch of computational science involved with finding the possible “best” solution to a problem [15]. The “best” here refers to satisfactory or acceptable solutions, from a set of possible candidate solutions. The overall best solution able to be found or not matter are determined by the characteristics and requirements of the problem. For example, the target search problems have always been associated with time constrain, and the solution required to be found within a limited time. In this specific situation, a good candidate solution may be enough. The objective of the targets search problems is to successfully search the available targets within the minimum time. There are similarities with the optimization and targets search problem in terms of their ultimate goal which is to find or search the possible available solution (target) to their problem. Figure 1 shows the publication trend of PSO strategies towards target search problems based on the Goggle Scholar database. The research article publication distribution shows the exponential trend which can be seen as evidence that interest in PSO strategies towards solving target search problems is continuously in demand. In line with this trend, Ismail and Hamami [16] in their systematic literature review outcomes also reveals that PSO has been the most preferred used algorithm by research communities in the target search problems.

Fig. 1

Goggle trend indicator of PSO strategies towards target search problems

The purpose of this study attempts to deliver a comprehensive systematic review analysis and inside view of the PSO strategies towards target search problems. It will be focused and dissected on the PSO components, target search components, and research field components related to the PSO strategies towards target search problems. The remainder of this article is organized as follows. Section 2 details the methodology section and the approach of the PRISMA Statement (Preferred Reporting Items Systematic Reviews and Meta-Analysis). Section 3 present an overview and characteristics of PSO. The 4th section systematically reviews and synthesizes the relevant research on PSO strategies towards target search problems by delivers the answers to the research questions. Open problems and future directions of the focus topic are addressed in Sect. 5. The article’s contributions are presented and concluding remarks are summarized in Sect. 6.

Methodology

In this section, the method used to systematically retrieve documents related to PSO strategies towards the target search problems is discussed. This article implemented the method know as PRISMA [17] and supported by Kitchenham [18] method to run the systematic review, eligibility confirmation and exclusion criteria, step of review process, and data extraction and analysis.

PRISMA

The PRISMA Statement (Preferred Reporting Items for Systematic reviews and Meta-Analyzed) was used as a guide for this systematic review. The PRISMA Statement has three main advantages compared to other systematic review methods [19]. There are:

The PRISMA method defines clear research questions that allow systematic research to be carried out.

The method identifies inclusion and exclusion criteria for the eligibility confirmation.

The method attempts to review a large database of scientific literature in a specified period.

With the above advantages of the PRISMA Statement, the rigorous search of terms related to PSO strategies towards the target search problems able to be carried out systematically.

Resources

This review extracts data from the world’s largest comprehensive academic search engine which is Goggle scholar with the estimation of 389 million records based on Gusenbauer [20] in Scientometrics journal. The results from the Goggle scholar then will be filtered with two establish high-indexed databases, Scopus and Web of Science (WOS) for the quality standard conservation purpose. Scopus is the largest abstract and citation database of peer-reviewed literature with over 25,000 journals from 7000 publishers across all continentals. On the other hand, WOS is a robust trusted publisher-independent citation database with over 100 years of a comprehensive backfile of the highest-quality research. It consists of 9200 impactful journals across 178 scientific disciplines in the Science Citation Index Expanded (SCIE). Both Scopus and WOS content is rigorously vetted and selected by an independent review board of experts in each field for the conservation of the quality standard.

Eligibility and Exclusion Criteria

There are four exclusions criteria are determined for the selected document's eligibility confirmation. The exclusions criteria are:

Works that are not included either in the Scopus or Web of Science (WOS) databases.

Works that are not related to the PSO and target search problems.

Works that only make a proposition without any experiment or comparison result presentation.

Works that are not presented or written in English.

Regarding the timeline, a period of two decades from 2004 until 2021 is selected which is a period where the PSO domains are starting to gain and evolved as the main strategy for solving target search problems.

Systematic Review Process

Based on the PRISMA statement there were four stages involved in the systematic review process. There are the identification stage, screening stage, eligibility checking stage, and selecting the included studies stage. The review process was conducted in August 2021.

Identification Stage

In the identification stage, identified keywords were used for the search process. The choice of keywords for construct the search string was relied on previous commonly found in the literature and the term that related to PSO strategies that were applied to target search problems. The selected specific keyword strings are: (“particle swarm optimization*” AND “target search*”). These keyword strings were run in the Goggle scholar database and 1073 searched documents emerged as the results.

Screening Stage

For this systematic review, there were two phases of the screening process. First, the identified searched documents (n = 1073) been filtered by documents that were only published either in Scopus or WOS databases (exclusion criteria E1). At this phase, out of 1073 documents eligible to be reviewed, a total of 967 documents were removed. Then the remaining 106 documents were checked for duplication. There were 35 duplicate documents between the Scopus and WOS databases, thus the result for the next eligibility checking stage was only 71 documents.

Eligibility Stage

In this stage, all 71 documents were filtered with the remaining exclusions criteria E2 until E4 as explained in Sect. 2.3. The total number of documents that have been excluded was 20 documents. The remaining 51 documents are eligible and included for the qualitative synthesis.

Included Stage

The final number of documents that have been selected for this systematic review was 51 documents. These documents are inclusive of research articles and conference proceedings. The flow diagram of the study is shown in Fig. 2.

Fig. 2

The flow diagram of the study

Data Abstraction and Analysis

Besides the PRISMA Statement method, this systematic review paper applied the method by Kitchenham [18] for data abstraction and analysis. The method required to carry out a formulated question for identification, selection and critically assess and analyze relevant data from studies that are included in the systematic review. The data (51 selected papers) have been systematically selected guided by PRISMA statement which has been explained in detail in Sect. 2.1 until 2.4.

There are three main considerations for the construction of the research questions. There are PSO components, target search components, and research field components. From these three components, there are nine research questions for nine considered elements have been constructed. The research questions are:

What variants of PSO are being used to perform the target search problem?

What kind of research field is PSO being implemented related to the target search problem?

What type of PSO inertial weight function is applied during the target search task?

What kind of mechanism is implemented to improve the PSO search result efficiency?

What type of PSO termination criteria that being carried out?

How many targets are available?

What is the mobility status of the available target?

What is the experiment framework that has been carried out?

What is the state of the search environment’s complexity?

The research question Q1 until Q9 will be the guideline for the data extraction and analysis. In line with the research questions, there are nine data extraction fields for this systematic review. There are:

Implemented PSO variants, being able to consider any PSO variants from basic to state-of-the-art variants.

Research fields that are related to the PSO strategy for the target search problem.

Type of PSO inertial weight function that is being applied during the target search task.

The mechanism that is implemented to improve the PSO search result efficiency.

Type of PSO termination criteria that are being carried out.

The number of targets that are being searched, either single or multiple targets.

Mobility of the target; either static or dynamic.

Experiment framework that has been carried out either simulation, experiment, or both simulation and experiment.

Search environment’s complexity; either obstacle-free or cluttered environment.

Background on Particle Swarm Optimization (PSO)

Particle Swarm Optimization (PSO) has been introduced as a method for optimization of continuous nonlinear functions by Kennedy and Eberhart [21] as one of the proceeding papers in the 1995 International Conference on Neural Network. The PSO mechanism has taken inspiration from bird flocking behavior simulation [22]. It is one of the Swarm Intelligence (SI) nature-inspired sub-domain under biological-based optimization algorithm along with ant colony optimization (ACO) [8], bacterial foraging optimization (BFO) [11], grey wolf optimization (GWO) [23], dragonfly algorithm (DA) [24], corona virus optimization (CVO) [25], and others. The PSO is categorized into a stochastic search strategy based on each particle's iterative interaction that forms the swarm.

A PSO mechanism works as a swarm of particles, where each particle represents a potential solution. A swarm considers as a population, while a particle is similar to an individual. The particles are explored through a multi-dimensional search space, where the position of each particle is dependent on its own experience and that of its neighbors. Each particle exploring a problem space for the best result (fitness) position within the search space. During the searching process, the particles will update their velocity and latest best position, and overall best position achieved within the neighborhood. The velocity (v) and position (x) will be updated for each ith iterations by below equations: andwhere and designated as velocity and position vectors for the ith particle, is the personal best position of the ith particle while gb is the overall best position of all particles within the neighborhood. ω is the inertia weight introduced to balance between the global search and local search by controlling how much the current velocity of the particle contributes to its velocity in the next iteration [26]. c1 and c2 are cognitive scaling and social scaling factors while rand1 and rand2 are random numbers drawn from a uniform distribution. A good and efficient PSO strategy should be able to maintain a balance between its global search (diversification) and its local search (intensification).

Since the PSO has been introduced, it has been applied to many applications and continuously has been evolved and improved. Most of the improvements and modifications are to improve its efficiency by improving the convergence of the PSO and increasing the diversity of the swarm. Four main aspects affect the efficiency of the PSO algorithm. These aspects are (1) particle initialization, (2) defining the term iteration, (3) function evaluation, and (4) stopping conditions. All the above aspects will be systematically analyzed in this review article.

Result of Systematic Review

Publication Distribution over The Years

One of the techniques to evaluate the research topic latest trend is by analyzing the publication distribution over the years. Figure 3 shows the number of articles that have been published (based on PRISMA Statement criteria in Sect. 2) from 2001 until 2020 in a five-year group. The articles were published in high-indexed, well-established databases, starting only with a single article that was published between 2000 to 2005. From 2006 until 2010 there were 15 publications which were about a 93% increase from the previous 5 years gap.

Fig. 3

The number of articles per year

There is a slight reduction by four publications between 2011 till 2015-year gap but the trend increased again by 54% between 2016 until 2020 with a total of 24 publications with an average number of articles of 3.2 articles per year. The increased percentage between the first-year group (2001–2005) and the last group (2016–2020) is about 96% which is evidence that interest in PSO strategies has been growing towards solving the target search problems. This increasing trend is supported by the increasing computational processing performance which would uncover a promising established research area in PSO strategies towards the target search problems.

Publication Distribution based on Publisher

The selected articles based on PRISMA method have been published in a wide range of publishers including publishers that are well established such as Springer, IEEE, and Elsevier. Figure 4 shows the publication distribution among the publishers. There are 24 journal articles and 27 proceeding articles from the total of 51 selected articles that are eligible to be analyzed. IEEE leading other publishers with 19 published articles (3 journal articles and 16 proceeding articles), Springer with nine published articles (6 journal articles and 3 proceeding articles), and Elsevier with three published journal articles. There are three publishers with each two published articles (SPIE Digital Library-proceeding, ACTA Press, and Inderscience-journal). Other 14 publishers have at least one publication for each of them as shown in Fig. 4. Most of the publishers covered the journals and conferences related to the application of computational intelligence and robotic in the engineering domains.

Fig. 4

Number of publication per publisher

Research Data Analysis

After a systematic analysis of the 51 selected articles, the review resulted in nine main elements related to PSO strategies towards target search problems. The nine main elements are PSO variant, application field (six application fields), PSO inertial weight function (four inertial weight function), PSO efficiency improvement (seven improvement methods), PSO termination criteria (four termination criteria selection), target available (two types of targets), target mobility status (two types of target mobility), experiment framework (three frameworks), and environment complexity (two types of environment complexity). The review result has been summarized in Table 1. This systematic review results presented a comprehensive analysis and inside view of the PSO strategies towards target search problems. All these nine elements have been extracted based on nine research questions that have been explained in Sect. 2.5

Table 1

The findings

Author	PSO variant	Application field						PSO inertial function				PSO efficiency improvement
		1	2	3	4	5	6	1	2	3	4	1	2	3	4	5	6	7
[27]	MTPSO	/							/					/
[28]	MEPSO	/						/					/
[29]	PSO	/						/				/
[30]	PSO	/							/			/
[31]	RbRD PSO	/							/				/
[32]	DPSO	/							/				/
[33]	E2R PSO	/							/				/
[34]	PSO	/						/				/
[35]	PSO					/		/										/
[36]	MFPSO	/							/			/
[37]	CFPSO	/						/						/
[38]	CPSO		/					/										/
[39]	IPSO		/					/				/
[40]	ATREL-PSO	/						/				/
[41]	LoPSO	/						/						/
[42]	RbRD PSO	/							/				/
[43]	PSO			/					/			/
[44]	PPSO	/						/				/
[45]	MO PSO	/						/				/
[46]	HPSO	/						/				/
[47]	AF-CAC PSO		/					/				/
[48]	A-RPSO	/							/				/
[49]	PSO	/								/								/
[50]	MPSO	/								/				/
[51]	PPSO	/							/			/
[52]	PPSO	/						/				/
[53]	MPSO	/						/				/
[54]	EPSO	/						/				/
[55]	IPPSO	/							/							/
[56]	PPSO	/							/			/
[57]	IPPSO	/							–							/
[58]	EPSO	/						/				/
[59]	GDME PSO	/						/				/
[60]	PSO						/	/				/
[61]	VL-ALPSO	/						/				/
[62]	PSO	/							/			/
[63]	PSO	/							/			/
[64]	OPSO		/					/							/
[65]	FPSO			/				/				/
[66]	DPSO			/				/				/
[67]	NMPP PSO			/				/				/
[68]	EPSO	/							/			/
[69]	PSO	/									/						/
[70]	EPSO	/						/				/
[71]	EPSO	/						/				/
[72]	NMPP PSO		/					/				/
[73]	EPSO			/				/				/
[74]	PSO	–						/				/
[75]	PSO	–							/			/
[76]	PSO				/			/										/
[77]	OPSO	–						/							–

1. AF-CACPSO = Adaptive Fusion Continuous Ant

Colony PSO

10. GDMEPSO = Group Decision Making

Extended PSO

25. VL-ALPSO = Velocity limit Augmented Lagrangian

20.NMPPSO = Non-Markovian

Process PSO

7. E2RPSO = Exploration Enhance Robotic

PSO Variant

The first element that this review article analyzed is PSO variants. There are a total of 25 variants of PSO have been revealed from the total of 51 research articles. Basic PSO leads the variants with 26% (13/51), Extended PSO (EPSO) with 12% (6/51), and Potential Field PSO (PPSO) with 8% (4/51) of the entire PSO variants. Modified PSO (MPSO), Repulsion based Robotic Darwinian PSO (RbRDPSO), Improved Potential Field PSO (IPPSO), Non-Markovian Process PSO (NMPPSO), Distributed PSO (DPSO), and Optimal PSO (OPSO), each covered 4% (2/51) of the entire variants. The remaining variants can be referred to Fig. 5 with only a single application.

Fig. 5

The PSO variants

The characteristics and mechanism of basic PSO have been explained in Sect. 3. The usage of basic PSO towards the target search problems can be traced back to research by Xiao et al. [76] for strategy implemented towards the magnetotelluric data inversion and interpretation. Research by Chen et al. [60] in medical image registration based on generalized mutual information and PSO-simplex search strategy, and the research of a new and practical PSO method in enhancing the undetectability of military camouflage by Lin and Prasetyo [35] also use basic PSO in their studies. The latest usage of the basic PSO can be detected in [29] which made the basic PSO is well received by researchers throughout these two decades in the target search problems area. Ever said that the improvement towards the basic PSO are progressively continued over time and one of the pieces of evidence is these reported 25 variants of PSO.

The second variant of PSO that attracts the attention of the previous researchers is extended PSO (EPSO). EPSO used the concept of cooperatively parallel control in research by Pugh and Martinoli [78] as the algorithm main strategy. The articles by Xue and Zeng also Xue et al. [70, 71] studied the comparison of a control method of swarm robots for target search in parallel and asynchronously with the basic PSO. The EPSO has been utilized for swarm robots’ interactions during the target search process. The same group of the researcher of Xue and Zeng [68], expanded their study of EPSO by introduced the kinematic constraints into the target search problem. EPSO successfully guided the swarm robots to complete the target search task within the constraints of its kinematic. The target position estimation aided swarm robotic search under conditions of relative localization mechanism has been introduced by Zan et al. [58]. In this research EPSO and Received Signal Strength Indicator (RSSI) have been used to estimate the target position.

One of the challenges in target search problems is been trapped into the local minimum. In potential field PSO (PPSO), potential field strategy has been used to provide swarm robots with the exploration priority characteristic thus leverage the local basic PSO strategy into global optimization method. A potential-PSO approach to cooperative target searching of multi-robots in unknown environments has been proposed by Cai and Yang [51, 56]. The proposed strategy first establishes a potential function according to the initial position of the obstacles, unsearched area, and targets. The PSO fitness function has been determined based on the potential function in the work area. The target search process then started with guidance by the proposed PPSO algorithm. The simulation results show that the proposed algorithm can complete the target search task in completely unknown environments successfully. PPSO also has been used in the study by Cao and Sun [44] which PPSO guided the target search and hunting process successfully regardless of the dynamic environment and the limitation of sensory and communication between the robots.

Table 2 portrays the comparison of major PSO variants that have been implemented in target search problems. Each variant has its advantages and limitations. From the comparison, it can reveal that each variant after the basic PSO will try to overcome the basic PSO critical limitation which is high in tendency to trap in local minimums. This limitation needs to be eliminated to guarantee success in accomplish the targets search task.

Table 2

PSO variant comparison

PSO Variant	Advantages	Limitation
Basic PSO	Search is implemented iteratively among particles with the consideration of each particle's best position for the selection of overall swarm position Algorithm implementation is simple and easy to understand Computational intensity is low	Not suitable for global search due to high tendency for trapped in local minimum Convergence to a local or global minimum are not guaranteed
Extended PSO (EPSO)	Task on hand is executed in parallel thus it can be done faster thus decreased the target search task time consumption Improving robustness against failure of single agents by redundancy as well as individual simplicity	EPSO only have been tested and suitable in obstacles free environment
Potential Field PSO (PPSO)	The percentage of robots to get trapped in local minimum is reduced Potential field strategy can offer evaluation of priority of undetected areas and help the swarm robots to search effectively	Have disadvantages to find the targets in complex environments (example: target that surrounded by obstacles.)

Application field

The target search problems using PSO have been applied in a wide range of application fields. Based on the systematic review results six application fields have obtained the PSO advantages in the target search problems. The research fields are swarm robotic, path planning/navigation, signal processing, image processing, data mining, and parameter optimization. The outcomes show swarm robotic field outclass other fields by 74% (38/51) follows by path planning/navigation and signal processing both with 10% (5/51). The other three research fields which are image processing, data mining and parameter optimization each covers 2% (1/51) of research fields that utilized the advantages of PSO in solving the target search problems. Figure 6 shows the proportion of the PSO target search application field.

Fig. 6

Proportion of PSO target search application

The data portray the swarm robotic application fields have obtained the most utilization of PSO in target search problems. The PSO basic mechanism of particles search for the best solutions in the search space is in line with the requirement of target search problems in the swarm robotic domain. The particle of the PSO can be mapped as the robots in the swarm and the best solutions in the PSO can be mapped as the search target in the target search problems. The swarm robotic application fields can be categorized into 3 categorized which are applied in general/land type of swarm robotic (84%), aerial swarm robotic (11%), and underwater swarm robotic (5%).

For the application in general/land type of swarm robotic, a proposal of a novel algorithm combining the Darwinian survival principle with ion-based repulsion mechanism for target searching problem has been presented by Dadgar et al. [79] The proposed algorithm combined the local search capability of PSO with ion repulsion-based concept for obstacle avoidance for static single-target search in a cluttered environment. The proposed repulsion-based robotic Darwinian PSO (RbRDPSO) performance has been benchmarked with the basic RDPSO, RPSO, and distributed PSO. The benchmark results portray the proposed RbRDPSO superiority compared with other methods in terms of speed and search results without the decreasing performance even the number of robots decreases.

Du [32] has been initiated a novel swarm robots targets search method guided by a distributed PSO (DPSO). The proposed algorithm considers the communication limit and the communication energy consumption (CEC) by introduced base stations deployment initiation and relocation. A dynamic swarm division strategy has been proposed during a static multi-target search in an obstacles-free environment to improve search efficiency. The simulation results show that the proposed algorithm outperformed other comparative algorithms in various simulation setup experiments.

Previous works by Kwa et al. [34] proposed the decentralized PSO-based strategy using a topological k-nearest neighbor graph with dynamic explore-exploit tunable. The proposed strategy implemented a dynamic PSO algorithm towards a dynamic single obstacles-free environment target search problem. The performance of the algorithm has been tested with a fully decentralized swarm of eight ground miniature robots. The experiment results validate the performance of the proposed strategy by successfully track and find the target.

In the aerial swarm robotic research fields, Gupta et al. [27] has introduced a decentralized UAV swarm method for multi-target search problems with the available onboard sensors information. The proposed method included three main mechanisms which time optimized multi-target search, optimized payload drops, and inter-UAV collision avoidance. The strategy utilized the modified PSO named the Multi-target PSO (MTPSO) as the proposed controller. The proposed strategy performance has been tested with various case scenarios using Ardupilot's SITL platform.

The studies of searching for moving targets using multiple unmanned air vehicles (UAVs) with sensing capabilities have been proposed by Hu et al. [46]. The proposed algorithm utilized the model predictive control (MPC) method hybrid with PSO crossover operator for path planning search optimization. The performance of the proposed algorithm has been validated using simulation and the cooperative search efficiency has been improved compared with the coverage search and the random search method.

In the last category of the swarm robotic which underwater swarm robotic field, Cao et al. [52] have been proposed a combination of potential field-based PSO (PPSO) with velocity synthesis algorithms multi-AUV target searching under ocean current influence. The proposed algorithm can be able to control AUVs swarm under the influence of ocean current while searching for the target. The potential function of the workspace determined the fitness function while the PPSO planned the reasonable paths iteratively. The strategy performance was evaluated through simulation. The simulation results validate the strategy is capable of offsetting constant as well as variable ocean currents with higher work efficiency and less energy consumption. All the above PSO usage in the target search problems example towards swarm robotic research fields portrays the robust, flexibility, and scalability of the PSO as a local (single target problem) or global (multiple targets) search strategy. The combination of efficiency improvements such as ion repulsion based, k-nearest neighbor graph, time optimized, and model predictive control, made the PSO the obvious choice of target search problems in the swarm robotic domain within the research communities.

The path planning/navigation research field also has a positive effect from the usage of PSO in the target search problems. Path planning enables the robot or agent to get from point a to point b with an optimized collision-free path [80]. Research by Ranaweera et al. [38] has been presented a new shortest path planning for firefighting robots based on the PSO algorithm. The proposed path planning strategy is capable of containing the fire spread by removing the flammable objects in their path. The PSO algorithm functioning to optimize the available possible path to produce the shortest safest path. The validation of the proposed algorithm has been done with Mathematica simulation software and the existence of the shortest path is proved.

A new navigation approach for two-wheeled robots for cooperative tasks carrying an object in unknown environments with concave maps has been introduced by Lai et al. [47]. The proposed navigation has three main elements which cooperative obstacles following (OBF), cooperative target searching (TS), and a behavior supervisor. The fuzzy controller executed the OBF element while the dead-cycle problem of navigation in concave maps has been avoided by the behavior supervisor. Target searching will continue to execute the target search task while the two other elements run their above task. The performance of the proposed navigation strategy was tested with simulations and experiments platform. The effectiveness of the navigation method in unknown environments with concave maps has been proved by the simulations and experimental results.

Signal processing is very important during extracting valuable information from different detectors sensors, and a subfield of electrical engineering includes analyzing, modifying, and synthesizing signals such as sounds, chemical properties, and radio wave signals [81]. The introduction of a self-organized hybrid wireless sensor network for finding randomly moving targets in an unknown environment has been made by Nighot et al. [43]. The proposed wireless sensor network used only mobile sensor nodes with a combination of static and mobile sensor nodes and the area is determined in advance. The proposed strategy has been implemented in MSNs Movement Prediction Algorithm (MMPA), Leader Selection Algorithm (LSA), Leader's Movement Prediction Algorithm (LMPA), and follower algorithm. The performance of the wireless sensor has been validated using a simulation platform which the result shows that hybrid (static and mobile WSN approach with less number of sensor nodes finds target faster than only MSN approach.

Image processing can be defined as the use of a computer to process different images using an algorithm [82]. One example of the usage of PSO related to target search problems in image processing is the proposal of a medical image registration based on generalized mutual information and PSO-simplex search strategy by Chen et al. [60]. The proposed algorithm objective is to overcome the problem of local extremum in target search that can affect the medical image registration process. Based on the obtained simulation results, rapid and accurate registration results are possible with the proposed algorithm.

The data mining field also gains advantages from the PSO application towards the target search problems. Research of the PSO application to the Nonlinear inverse problem of magnetotelluric sounding data has been studied by Xiao et al. [76]. The PSO has been selected in the proposed strategy to be implemented towards the magnetotelluric data inversion and interpretation. The PSO numerical tests have been carried out using MATLAB and the performance of the PSO has been tested using theoretical and observed magnetotelluric (MT) data. Based on both tested results, the PSO algorithm is not dependent on the initial models and can obtain the global optimal solution.

Parameter optimization can be considered as a field of problems that pursue optimal control parameters [14]. The study of a new and practical PSO method in enhancing the undetectability of military camouflage has been proposed by Lin and Prasetyo [35]. The proposed PSO strategy generates newly proposed camouflage as an empirical parameter based on the lower and the upper bounds from selected four different colors in swamp background. There are eight different locations (20 × 50 pixels) in the one swamp background that were selected to be the place of a human-shaped target. The CIELAB color space (L*a*b*) was initiated with the purpose of color differences specification and is one of the most utilized color spaces. L* represents the perceived lightness where the true black will score a zero L* and the perfect white would carry an L* of 100 in the CIELAB color space. Coordinate a* associate with red if + a* and green for −a* while coordinate b* correlates with yellow for + side (+ b*) and blue for -side (-b*) in the CIELAB color space. The optimum shift of %L*, %a*, and %b* from the original to the empirical parameter is adjusted by the PSO predictive algorithm. The evaluation of the original and newly proposed camouflages has been evaluated by 30 participants. The t-test experiments results indicate that the newly proposed military camouflage had a significantly lower camouflage similarity index value with a longer detection time.

PSO Inertial Weight Function

The performance of the PSO algorithm target search ability depends mostly on the trade-off between global (exploration) and local (exploitation) of each particle in the swarm. One of the aspects that affect the PSO performance is the selection of inertial weight (ω) function. Inertial weight function has been introduced by Shi and Eberhart [83] to control the exploration and exploitation abilities of the swarm by weighing the impact of the particle previous velocity on the current velocity. In normal practice, the inertial weight function will be between the range of 0 and 1 value (ω = [0,1]). Large value of ω will promotes exploration search while small ω facilitate local exploitation search. ω value selection also heavily depends on acceleration constants c1 and c2. Eberhart and Shi [84] in their empirical study, found the value of ω = 0.7298, c1 and c2 = 1.49618 work well in general for the PSO algorithm but it also noted that inertial weight function are problem-dependent parameters. From the review data there are four types of inertial weight function that been used in the previous research on the target search problems. There are (1) constant, (2) random inertial weight, (3) squared decreasing and (4) sigmoid increasing inertial weight function. Table 3 shows the mechanism, suitability, and mathematical function of each inertial weight function.

Table 3

The mechanism, suitability, and mathematical function of inertial weight function

Function	Mechanism and Suitability	Mathematical Function
Constant	Have manual control over the exploration–exploitation criteria Large value ω facilitates exploration with increased diversity while a small ω promotes local exploitation	ω=[0,1]
Random Inertial Weight	Suitable for a situation with a large number of targets Continuously exploits the data	ω=0.5+Rand()2
Squared Decreasing	Suitable for a moderate number of targets High exploration in the early stages then gradually decreased over time	ω=ω_max-(ω_maxω_min).iterationiteration_max^2
Sigmoid Increasing	Suitable for a small number of targets High exploration till no targets is found, exploration decreased when targets appear	ω=(ω_start-ω_end)(1+e^10(log(gen)-2)).(k-n.gen)+ω_end

Review analysis revealed a majority of 61% (31/51) of works in this domain applied the constant function of inertial weight in their PSO algorithm. The result reflected that the researchers choose to have full control over the exploration and exploitation trade-off by heuristically decide the suitable constant function value for each problem. The data also showed 33% (17/51) of research, selected to used squared decreasing inertial weight function in their algorithm. The search process pursues a high level of exploration in the early stages then gradually decreased over time. There are two articles [49, 50] that implemented the sigmoid increasing function as their inertial weight function. The search process in the sigmoid increasing function proceeds with the high-level exploration till all the targets have been found and starting to decrease once the first target appears. The review data also revealed that there is a single paper [69] that the proposed algorithm able to execute either one of inertial weight functions which are squared decreasing or random initial weight function.

PSO Efficiency Improvement

Besides the inertial weight function, population diversity, parameter adjusting, and hybridization method also have direct implications towards PSO search efficiency improvement. Based on the review data analysis, there are seven improvement methods which are a standard method, hybridization method, three population diversity methods (new operator, sub swarm, and reinitialization), and two parameter adjusting methods (dynamic tuning and fuzzy logic).

From the systematic review Table 1, there are 63% (32/51) of the studies merged (hybridization) the PSO strategy with others such as topological k-nearest neighbors [85], fruit fly optimization algorithm (FOA) [86], and potential field method [87]. Population diversity of new operator was the second most popular improvement method with 12% (6/51) and population diversity of sub-swarm method and standard method in the third place with the same percentage of 8% (4/51) each. Population diversity of reinitialization and parameter adjusting of dynamic tuning each with 4% (2/51) and the least popular of PSO efficiency improvement method among the previous researchers is parameter adjusting of a fuzzy logic method with only the study by Grant and Venayagamoorthy [64] applied the method.

Hybridization's main objective is to use the advantages of one strategy to overcome the limitation of another strategy. In this review context, the limitation of PSO strategy which high tendency towards trapping in local minimum needs to be addressed in conjunction to achieve a trade-off between exploration and exploitation. Figure 7 previews the choices of hybrid strategies with PSO in previous studies.

Fig. 7

The choices of Hybrid strategies with PSO in previous studies

22 strategies have been used in the previous studies that are compatible with PSO for the hybridization method. Time-varying Character Swarm (TVCS) is the most utilized strategy with five previous research. The research by Xue et al. [71] presented the TVCS concept for signal comparison and decision making on the best-found position. The TVCS concept can extend the communication range of each robot beyond its communication limitation due to its hardware and power supply constrain. This characteristic directly expands the horizon of PSO search capability during the target search task. Other strategies as shown in Fig. 7 have them each adding value towards the PSO strategy for search efficiency improvement such as potential field strategy in [52] enable the obstacle avoidance capability and topological k nearest neighbor in [29] provide the search and track capability towards the dynamic target.

The population diversity method also can improve the efficiency of PSO. Based on the systematic review result there are three mechanisms for maintaining the population diversity which introduces new operator, sub-swarm, and reinitialization. Research by Yang et al. [33] introduced the addition attraction term as the new operator. This attraction term functions as obstacles avoidance and unexplored regions search guidance. This process increasing the swarm's task-specific (top-down) diversity thus avoid the algorithm falling into local optimums. The proposed algorithm by Du [32] considers the communication limit and the communication energy consumption (CEC) as the new operators by introduced base stations deployment initiation and relocation. These new operators improve the search efficiency in multi-target search environment. The sub-swarm or multi-population strategy concept where the main swarm is divided into several subpopulations to enhance the exploration capabilities of the PSO algorithm. Sub-swarm strategies are relatively easy to integrate with PSO algorithm and the efficiency is better than a single population PSO algorithm [88]. The study in [37] grouped the robots after several iterations of stochastic movements and was able to change dynamically with each group specified to search only to a single target. The proposed sub-swarm strategy shows the adaptability, accuracy, and efficiency in multi-target searching over other previous methods. The last mechanism for maintaining the population diversity is the reinitialization of the particles. This method is an efficient way for facilitates the exit of particles from local minimums. The optimal PSO for collective robotic search applications by Doctor et al. [77] utilized the reinitialization of the particles by using a secondary PSO over the main PSO. The initial PSO particles are been reinitiated in the secondary PSO. By implementing the reinitialization method, the simulation results show improvement for both single and multiple target searches.

Parameter adjusting methods of dynamic tuning and fuzzy logic also contributed to improving the efficiency of PSO in target search problems. Research by Cai and Yang [57] take the advantages of dynamic parameters adjusting to improve the task completion of target searching by the swarm robots while Venayagamoorthy et al. [69] have been proposed the PSO strategy with the fuzzy logic controller implemented in landmine detection and firefighting using swarm robots. Both studies utilized the parameter adjusting method to improve the PSO algorithm efficiency.

PSO Termination Criteria

The last aspect that affects the efficiency of the PSO performance is the stopping condition or termination criteria of the iterative search process. Two considerations need to be concerned when selecting the termination criteria [15]. which are:

The selected termination criteria should not contribute towards the premature converge due to entrap into a sub-optimal solution.

The termination criteria should not contribute towards unnecessary excess of computational intensity due to fitness oversampling.

Based on the review analysis of the PSO termination criteria, there are three types of stopping conditions that have been implemented. There are:

Terminate when the target has been found.

Termination criteria no 1 have a direct relation with the objective function (fitness function) of the algorithm. If the fitness value has achieved its setting threshold the search process will be terminated. The research articles in [36, 41, 79] have been setting the fitness value (ƒ(x) > 0.99) as its termination criteria threshold.

Terminate when a maximum number of iterations has been exceeded.

The above termination criteria terminate the research process when the setting of the maximum number of iterations has been completed. There is no specific number of maximum iterations that fit all the target search problems but for the statistically steady conditions, research by Kwa et al. [34] and Grant and Venayagamoorthy [64] setting the maximum iterations by 500 iterations.

Terminate when there is no improvement is observed over a number of iterations.

The type 3 termination criteria also do not have one number fit all solution. For reference, an article by Dadgar et al. [48] set the maximum iteration without any improvement to 5 iterations.

There are 49% (25/51) of research articles that used termination criteria based on targets have been found. Most of the research that has been used this type of termination criteria are the research that set their target quantity to a single target search problem where the target search task completion is a must. The termination based on maximum iterations has been selected by 35% (18/51) of the previous researcher in this domain. The type of research that utilized this termination criterion is usually related to time restriction target search problems such as [89] and [90] in search and rescue missions. There is a case where a combination of the termination criteria is needed. This is because the researchers wanted to gain all the advantages of all selected termination criteria. 14% (7/51) of research articles selected both termination criteria based on targets that have been found and based on maximum iterations. Only a single research article [48] has chosen all three types of termination criteria to be implemented in their PSO algorithm. Table 4 shows the comparison (advantages and limitations) of PSO termination criteria.

Table 4

Comparison of PSO termination criteria

Termination criteria	Advantages	Limitation
Termination criteria based on targets have been found	Suitable for a problem such as training neural networks problem where the optimum is known initially which is usually zero	The setting of the error threshold needs to be selected with care. If too large the search process will terminate on a sub-optimal solution and if too small the search will endlessly searching without terminate at all
Termination criteria based on maximum iterations	Able to force termination if the algorithm fails to converge to protect fitness oversampling issue Suitable for time restriction target search problems such as search and rescue missions	If the setting of the maximum number of iterations is too small, termination could happen before a good solution has been found
Termination criteria based on no improvement over a number of iterations	Able to force termination when the average velocity over a number of iterations is approximately zero or only small position updates are made to protect fitness oversampling issues	Not always help to improve the algorithm performance due to an optimum is not necessarily reached when positions no longer improve

Target Availability and Target Mobility Status

Target quantity and target mobility status are important parameters for the target search experiment design. There are two types of targets available which single target and multiple targets. Data from Table 1 disclosed that 63% (32/51) of the research articles set their experiments with a single target over multiple targets. In terms of target mobility status, there are static and dynamic types of target mobility status. Review data shows 82% (42/51) of the research set the target mobility to the static conditions. These single static target research trends, support the fact that the researchers focused on emerging PSO swarm behaviors and interaction between the search particles during the target search task.

Experiment Framework and Environment Complexity

Finally, the review analysis also revealed that 92% (47/51) of previous research used the simulation platform to verify the PSO algorithm over the real experiment. The possibility to test a vast number of agents without consuming a large budget and time-consumption matter are two main reasons why the simulation platform is popular within the researcher’s community. The final data that were revealed was the target search task environment complexity. A total of 63% (32/51) of research articles set the environment as obstacle-free while the remaining research articles which 37% (19/51) prefer the cluttered environment as their target search task environment.

Open Problem and Future Directions

There is no doubt that research on PSO towards the target search problems application has advanced a lot in recent decades. Ever said that there are several problems remain open and become the future directions of these research domain. The open problems and future directions in the study of PSO towards the target search problems are addressed as below [14, 91, 92]:

The establishment of the optimal and sweet spot balance between exploration (diversification) and exploitation (intensification). This balance between diversification and intensification can provide a high level of efficiency towards the PSO algorithm.

Focus on promoting the overall optimization performance of PSO algorithm by characteristic refinement of an individual agent. Some initial step has been taken by able each agent to switch interaction patterns depend to the situation in hand.

Development of a unified mathematical model structure for analyzing PSO performance in terms of its complexity, stability, convergence, and parameters fine-tune.

Development of a test environment that can simulate and replicate the real-world constraints (dynamic environments) with a comprehensive set of metrics for PSO performance evaluation.

Conclusion

This systematic review has highlighted the PSO utilization towards target search problems by systematically retrieved documents using PRISMA statements and answering the research questions. The publication's exponential trends over these two decades supported that the PSO strategies have been the main selection in solving the target search problems.

The review focused an in-depth analysis of nine main elements related to PSO strategies towards target search problems which are PSO variants, application field, PSO inertial weight function, PSO efficiency improvement, PSO termination criteria, target available, target mobility status, experiment framework, and environment complexity. These nine elements are based on three main considerations of PSO components, target search components, and research field components. For the PSO component, basic PSO still gains the most popular among the previous researchers because of its algorithm implementation simplicity. Regarding the PSO inertial weight function and PSO efficiency improvement, the constant weight function and hybridization method remain the popular choice. Target has been found criteria gained the number one selection for the PSO termination criteria. The result in the target search component reveals most of the previous research set up their research with the single and static target in the obstacles-free environments and verified it using a simulation platform. The swarm robotic leads other research fields for obtained the most utilization of PSO in target search problems.

From the addressed summarized results, it should be acknowledged that PSO strategies have been successfully applied to solve the target search problems across several research fields. However, there are still gaps for further improvement which have been explained in the previous section. The research of PSO towards target search problems is still active, its potential by far from being saturated and will continue to increase in the future.