Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review

It has been proven in the literature beyond any doubt that meta-heuristic optimization algorithms perform well by optimally handling several versatile real-world optimization tasks, ranging from robotics [31], wireless networks [195, 206], power systems [40, 76], job shop scheduling [109, 137], to classification [35] and training of artificial neural networks [83]. While questing the global best (near-optimum) solution, numerous fitness evaluations are required by the meta-heuristic algorithms. This typically poses a serious barrier against the application of meta-heuristic algorithms to high-computational optimization problems that extensively exist in computational fluid dynamic optimization [21] and structural optimization [79], among many others. For working out those problems, the performance of candidate solutions is usually evaluated with high-fidelity numerical analysis approaches (e.g., computational fluid dynamics simulations or finite element analysis), which may deduct CPU time from minutes to hours, or even days [80, 222]. Therefore, within the paradigm of meta-heuristics, many Swarm Intelligence (SI) techniques and variants were proposed to tackle complex/large-scale optimization problems.

SI, a substantial branch of optimization techniques, is the characteristic of a system where agents interact locally with their environment so that their collective behaviors render the emergence of cohesive functional global patterns. Unlike Evolutionary Algorithms (EAs), SI techniques are inspired by agents’ plain behaviors and self-organizing interactions, such as fish schooling, honey bees, bacterial growth, animal herding, bird flocking, ant colonies foraging, and so on. Indeed, Beni [17] was the first to use the SI term in cellular robotic systems, in which simple agents interact with neighbors to organize themselves. SI was formally established in [18, 19]. The rife SI algorithms include Ant Colony Optimization (ACO) [37] and Particle Swarm Optimization (PSO) [91]. Less widespread SI algorithms are Bacterial Foraging Optimization (BFO) [141], Artificial Bee Colony (ABC) [87], Firefly Algorithm (FA) [201], and many others. SI algorithms were primarily developed for steady optimization problems. However, dynamic environments involve several real-world optimization problems [124].

Typically, a swarm is defined as a vast number of simple, homogeneous agents interacting locally with their environment, as well as themselves, with decentralized control to authorize the emergence of a global important behavior. Swarm-based techniques have recently arisen as a family of swarm-based, nature-inspired algorithms that have the ability to produce robust, fast, and low cost solutions to numerous complex problems [50, 69]. Therefore, SI can be figured out as a major category of Artificial Intelligence (AI) that is utilized to model the collective behavior of natural social swarms, such as honey bees, bird flocks, and ant colonies. These agents (swarm individuals or insects) are relatively gullible with simple own capabilities. However, they perform cooperatively tasks substantial for their survival through interacting together in particular behavioral manners. Socially, swarm individuals can directly or indirectly interact among themselves [202]. Direct interaction can be through audio or visual contact (e.g., a waggle dance of honey bees), while indirect interaction is evident when the environment is changed by one individual and the other individuals react to the new environment (e.g., pheromone tracks of ants that look for food sources through depositing on their way). This indirect pattern of interaction is known as “stigmergy”, which denotes communication through the environment [42]. The research area presented in this thorough paper focuses on SI. More specifically, this review strives to explore one of the most popular models of SI, PSO, which is inspired by birds’ flocking behavior.

PSO is a swarm-based stochastic algorithm proposed originally by Kennedy and Eberhart [44, 91], which exploits the concepts of the social behavior of animals like fish schooling and bird flocking. In PSO, each potential solution to a given problem is viewed as a particle with a certain velocity flying through the space of the problem just like a flock of birds. Each particle then combines – with some random disturbances – some aspect of the record of its own historical best location and current location with those of one or more agents of the swarm to determine its next movement through the search space. After all particles have been moved, the next iteration occurs. The swarm as a whole (e.g., a flock of birds collectively searching for food) is probably to gradually approach the objective function optimum. PSO has eventually gained prevalent vogue amongst researchers and emerged to provide high performance in an assortment of application areas, with the potential to hybridize and specialize and demonstrate some appealing emergent behaviors. PSO has a main advantage of having fewer parameters to tune. PSO obtains the best solution from particles’ interaction, but through high-dimensional search space, it converges at a very slow speed towards the global optimum. Moreover, in regard to complex and large datasets, it shows poor-quality results. If there is a large number of dimensions in the problem at hand, PSO usually fails to discover the global optimum solution. This phenomenon is caused not only by the presence of local optima trap, but also the potential fluctuation of the velocities of particles such that the successive range of trials is bounded within a sub-plain of the whole search hyper-plain [178].

The local optima issue in PSO has been discussed and diverse variants of PSO algorithm were developed to tackle that issue. For example, some of these variants have been developed by incorporating the capabilities of EAs (e.g., the adaptation of PSO parameters, hybrid versions of PSO, etc.), thereby producing adaptive PSO versions. Other researchers have taken on incorporating evolutionary operators, such as crossover, mutation, selection, as well as the Differential Evolution algorithm (DE) itself, into PSO. Consequently, hybrid versions of PSO have been tested and produced, including the hybrid evolutionary PSO [128], Genetic Algorithm (GA) and PSO [48, 116], genetic programming-based adaptable evolutionary hybrid PSO [154], and many others [62]. Such improvements perform well with PSO and have the potential to avoid getting stuck at local optima. However, the problem of premature convergence in some high-dimensional complex problems still exists, even if the local optima obstacle is absent. Hence, PSO does not always work properly for high-dimensional models [54].

To this point, the performance of PSO has generally been improved by developing different variants of the algorithm. However, few review papers and technical surveys have systematically addressed the PSO literature [190, 216]. Consequently, the primary objective of this work is to present a systematic survey by reviewing the PSO algorithm and its different methods, as well as a wide variety of indicative application domains. Moreover, some of the addressed studies have highlighted the main approaches of PSO applications, including health-care, environmental, industrial, commercial, smart city, and general aspects. The key solicitude of those PSO methods and applications is satisfying quality standards: services arising from PSO methods and applications should support user’s requirements that cover the quality metrics of accuracy and efficiency, such as convergence rate, computational cost, consistency, stability, diversity, etc. In other words, this paper aims to present a timely, compendious, systematic and an in-depth overview of the PSO algorithm between 2017 and 2019 and the opportunities and challenges imposed during this period.

The structure of this study is organized as follows. Section 2 presents the related works. A brief foundation of the PSO is presented in Sect. 3. Section 4 exploits the Systematic Review (SR) procedure to provide the proposed research approach and motivation. Section 5 demonstrates the various methods of PSO based on the SR method. Section 6 systematically outlines and categorizes the application approaches in PSO. Also, a technical classification and differentiation of the approaches in the addressed papers is presented in this section. Section 7 provides a discussion on the PSO approaches that have not yet been thoroughly analyzed. Finally, Sect. 9 concludes the paper along with its limitations.

A swift explanation is presented in this section for the general related studies in the PSO algorithm.

Poli et al. [143] presented an overview of the great efforts which have given impetus and direction to research in particle swarms, as well as some important new applications and directions. An analysis of IEEE Xplore and Google Scholar citations and publications from 1995 to 2006 were presented in this work, illuminating the sense meant by Kennedy and Eberhart [92]. The strength of this study was to present comprehensive challenges and open issues in the PSO algorithm. However, this study did not consider the compatibility of PSO application with each presented approach.

Banks et al. [13, 14] offered, in two parts, a timely and brief review of the field in general, alongside the opportunities and challenges emanating from the versatile application of PSO. On the one hand, part I [13] has considered the history and background of PSO and its position within the broader paradigm of natural computing. The review then continued to discuss different improvements to the native formulation of PSO both in discrete and continuous problems, swarm behavior analysis, and measures considered to address stagnation. Furthermore, the review focused on research regarding adaptations for parallel implementation, algorithm configuration, and dynamic environments. The achievement of this study was identifying two significant areas of challenge for future further development: swarm stagnation and dynamic environments. The shortcoming of this part is the insufficiency of the explanation the related work. On the other hand, part II [14] has discussed recent studies in some of the impressive areas of research: constrained and multi-objective optimization, combinatorial problems, and hybridization. In that review, a number of domains were briefly touched upon, including the optimization of neural fuzzy networks, artificial neural networks, computational biology, image processing and medical imaging, optimization of electricity generation, network routing, and financial forecasting. Nevertheless, the study has a main defect of not analyzing the selected studies in terms of evaluation parameters, such as convergence rate, diversity, accuracy, and processing time, as quality factors in this area.

Kulkarni et al. [96] outlined some issues regrading Wireless Sensor Networks (WSNs), introduced PSO, and discussed the suitability of the algorithm for WSN applications. This study also presented a compendious survey of how PSO is tailored to tackle the issues of node localization, node deployment, data aggregation, and energy-aware clustering as optimization problems. The strong point of this review is to comprehensively present open issues in WSNs. However, the compatibility of high-speed real-time applications with each approach was not considered.

Kothari et al. [94] reviewed the studies related to the modifications of native PSO and its practical application in real-world problems. The rapid modification of PSO has been emerged in different examples, including the two-step PSO and the PSO-Support Vector Machine (PSO-SVM). The integration of the PSO and the practical implementation of PSO with the industry standard algorithm have also generated stunning results. The advantage of this survey was to present recent diverse variations in PSO and analyze the accuracy of PSO in different domains. The main deficiency of this study is the lack of statistical information about the discussed standard PSO and its application in various specified contexts.

Imran et al. [71] discussed diverse variants of PSO in regards to swarm initialization, mutation operators, and inertia weight. The main advantage of this overview was to highlight the importance of introducing the different mutation operators and inertia weight parameter to improve the performance of PSO. However, other promising variants of PSO were not considered.

Alam et al. [7] surveyed systematically the evolution of clustering techniques based on PSO and presented the results of rapidly increasing trends in the literature of SI, PSO paradigm, and PSO-based data clustering approaches, proving that there is a tremendous increase in the popularity of such approaches. This research affirmed that the approaches are novel and simple to implement and builds on communication and collaboration. This study outlined different application domains of PSO relevant to clustering. However, applications to more complex problems are absent. Furthermore, variants of PSO for clustering high-dimensional data were surveyed by Esmin et al. [49]. The main advantage of this survey was to present the attempts of different publications in reducing data dimensions. The weakness is still the absence of discussing more clustering-related applications.

Marini and Walczak [123] described comprehensively the PSO algorithm to show its ability to solve different kinds of optimization problems in chemometrics. The importance of the appropriate choice of PSO meta-parameters has been highlighted in this work by means of selecting practical examples in the subject areas of variable selection, estimating robust Principal Component Analysis (PCA) solutions, and signal warping. This tutorial contributed to presenting works subject to chemometrics impressively. However, it lacked splashing other state-of-the-art fields.

Zhang et al. [216] investigated comprehensively the PSO algorithm. They provided advances in PSO, including its theoretical analysis (convergence analysis, parameter tuning, etc.), hybridization (with GA, DE, ABC, ACO, biogeography-based optimization, harmonic search, Tabu Search (TS), Artificial Immune System (AIS), and Simulated Annealing (SA)), modifications (including fuzzy PSO, chaotic PSO, bare-bones PSO, quantum-behaved PSO), population topologies (including star, ring, random, von Neumann, fully connected, etc.), extensions (to binary, discrete, constrained, and multi-objective optimization), and parallel implementation (in cloud computing, multiprocessor, multicore, and GPU forms). Moreover, they introduced a survey on the applications of PSO to the following eight fields: biology, chemistry, medicine, electrical and electronic engineering, fuel and energy, mechanical engineering, operation research, communication theory, and automation control systems. Despite all that, the publication analysis demonstrated in this survey lacks the presentation of the annual exponential fluctuation of publications for each variant and application field.

Wang et al. [190] presented the inception and background of the PSO algorithm, and carried out a theoretical analysis of the algorithm. Then, they analyzed its current situation of application and investigation in algorithm structure, topology structure, parameter selection, multi-objective optimization, discrete and parallel PSO algorithm, and engineering applications. This overview is characterized by suggesting distinctive future research directions. However, no analytic discussion is introduced in this work.

Summarizing, Table 1 outlines the recent review studies on PSO with respect to the general survey and review studies discussed in this section. This table lists the key topics, publication year, and covered years (if available) for each study. The deficiencies in the existing review papers propose that a comprehensive and systematic literature review should be provided to address some of such common weaknesses as:

• In existing studies, the methods and applications of PSO are not organized in a clear taxonomy, and do not have analytical assessment from which one can learn something beneficial.

• Important assessment factors are not involved in the methods and applications of PSO in some current studies.

• The systematic arrangement is typically absent in the structure of literature studies, and the paper selection criterion is vague as well.

3.1 Self-Organization Features

SI system has a major feature, namely, self-organization, in which the components of an initially disordered system interact locally to produce a coordination or global order. This process is characterized by spontaneousness; that is, no agent inside or outside of the system dominates the interaction. The self-organization in swarms was interpreted by Bonabeau et al. [25] through three key components as follows:

• Robust dynamical non-linearity (always comprising positive and negative feedback) convenient structures are promotionally being created with the help of positive feedback, while this positive feedback is counterbalanced and the collective pattern is stabilized with the help of negative feedback.

• Trade-off between exploration and exploitation A valuable mean creativity artificial approach is provided through a suitable balance that is identified by SI.

• Multiple interactions Information coming from neighbor agents in the swarm are used by individual agents, allowing information to be disseminated throughout the network.

3.2 SI Features

Millonas [127] proposed that SI must satisfy five basic principles: adaptability, diverse response, stability, quality, and proximity. Table 2 lists their meanings.

3.3 Standard PSO Algorithmic Structure

A swarm of particles updates their relative positions from iteration to another, boosting the PSO algorithm to duly perform the search process. To get the optimum solution, each particle moves towards its prior personal best position (\({\mathbf {p}}_{best}\)) and the global best position (\({\mathbf {g}}_{best}\)) in the swarm [215]. Assuming a minimization problem, one have

where \(i\in \{1,2,\dotsc ,N\}\), and

where i denotes particle’s index, t is the current iteration’s number, f is the objective function to be optimized (minimized), \({\mathbf {x}}\) is the position vector (or a potential solution), and N is the total number of particles in the swarm. The following equations update, at each current iteration \(t+1\), the velocity \({\mathbf {v}}\) and position \({\mathbf {x}}\) of of each particle i as:

where \({\mathbf {v}}\) represents the velocity vector, \(\omega\) is the inertia weight utilized to balance the local exploitation and global exploration, \(\mathbf {r_1}\) and \(\mathbf {r_2}\) are random vectors uniformly distributed within the range \([0, 1]^D\) (D being the search space dimensionality or the size of the problem at hand), and \(c_1\) and \(c_2\), called “acceleration coefficients”, are positive constants.

An upper bound is commonly set for the velocity vector. As a means to prevent particles from shaving off the search space and forcing them to take a proper step size to comb the entire search domain, the “velocity clamping” method was used [163]. The “constriction coefficient” strategy is another method, proposed by Clerc and Kennedy [36], in which the velocities can be also constricted by theoretically observing and analyzing the swarm dynamics.

By scrutinizing Eq. (3), we can interpret that the first part, known as “inertia component”, represents the prior velocity, which provides the particles with appropriate momentum to rove across the search space. The second part, the “cognitive component”, denotes the own positiveness for every particle. It motivates the particles to move towards their own best positions found so far over subsequent iterations. The third part, known as the “social component”, indicates the collective effect of the particles to reach the global optimum solution [218].

3.4 PSO Pseudocode

Let \(f : {\mathbf {R}}^N \rightarrow {\mathbf {R}}\) be an objective function which needs to be simplified. Then, the function takes a vector of N real numbers denoting an N candidate solutions and outputs a real number that indicates the value of the objective function. The f gradient is either hard to calculate or often unknown. Then, the global minimum \(\left( \mathbf{g}_{best}\right)\) is pursued as exhibited in Algorithm 1.

3.5 PSO Flowchart

The flowchart to demonstrate the flux of PSO is depicted in Fig. 1.

The flowchart of PSO

Full size image

This section depends on the SR method to present, in a taxonomy, a review of the research studies related to the methods and applications of PSO [71, 123, 143, 190, 216]. Other synonyms and alternatives of the key essential components are also considered to commit the final exploration string as [49, 94, 96]:

• (“Methods” OR “Variants” OR “Hybrid” OR “Improved” OR “Software” OR “Application” OR “Application layer” OR “Application-based” OR “App”) AND (“PSO”) OR (“Particle Swarm Optimization”)

Figure 2 demonstrates the distribution (from 2017 to 2019) of research studies published by the most popular venues regarding the review methods and articles, including Elsevier, Springer, IEEE, ACM, MDPI, Taylor & Francis, Wiley, Inderscience, MDPI, etc.

Distribution of research papers by publisher

Full size image

In relation to the objectives of this SR paper, comprehensive answers are provided to the following four Analytical Questions (AQs) [156]:

• AQ1: Into which domains can PSO methods and applications be classified?

• AQ2: Which main contexts can be considered for PSO methods and applications?

• AQ3: What assessment environments are used for evaluating the PSO methods and applications?

• AQ4: What are generally the PSO open perspectives and future research directions that can be drawn from the work presented in the paper?

After outlining the AQs, the ultimate research selection was conducted by applying the inclusion/exclusion criteria. With respect to the number of published research papers, only conference papers and journal articles indexed in Web of Science (WoS) or Scopus proceedings are selected and analyzed as peer-reviewed publications for the methods and applications of PSO. Finally, I selected 2,140 peer-reviewed papers. Due to the length limitation of the paper, I addressed curtly only 84 state-of-the-art research papers from reputed journals to address the four AQs in detail in Sects. 5 and 6. Figure 3 demonstrates the selection criteria and evaluation method for the literature studies. The exclusion phase was carried out by omitting white papers, short papers, book chapters, non-peer-reviewed studies, and low-quality researches (published in rapacious journals) that did not provide any technical information or scientific discussion. The ultimately selected studies are recognized by considering the inclusion criteria as follows:

• Papers published online from 2017 to 2019.

• Papers on PSO topics, including methods and applications.

• Papers prepared in a technical quality method in PSO methods and applications.

• Papers that follow the quality standards.

The ultimately selected studies are recognized by considering the exclusion criteria as follows:

• Papers that present survey and review studies.

• Papers indexed in neither WoS nor Scopus.

• Papers written in a language other than English.

• Papers not subject to peer-review.

The selection principles and evaluation of research papers

Full size image

In fact, the literature is rich in many PSO techniques. To organize them, a proper classification criterion is suggested, and various PSO methods are classified as per Fig. 4.

Categorization of diverse PSO methods

Full size image

5.1 Hybridization

Hybridization is a generic model of two or more algorithms, which exploits their advantages while decreasing their impediments. The combination of techniques does well for tacking a specified problem, given that the obtained results can be improved by these combined techniques on their own. The exploitation and exploration of an algorithm can completely be enhanced through hybridization of algorithms [53, 177, 180]. For example, an algorithm can cater for the lack of its preciseness and refine the results through synergy with a local search method. The hybrid approach in the field of optimization is growing in popularity and utilizes the purport of hybridizing the components from leading optimization techniques to improve the performance of traditional optimization algorithms.

5.1.1 Meta-heuristic Algorithms

Many nature-inspired hybrid approaches have been developed by numerous researchers/scholars for achieving significant performance in the exploitation and exploration of existing algorithms. To optimally design a Labyrinth Spillway (LS) having quarter-round or half-round crest shape, an evolutionary Hybrid Algorithm (HA) combining the PSO algorithm with the Bat Algorithm (BA) was developed by Ferdowsi et al. [52]. This way, the best outcomes of one algorithm substitute the worst ones from the other. Moreover, the proposed algorithm has a high convergence rate regarding the optimal absolute outcome and does not get stuck into the local minima. Responses and computational time of this HA proved that LS with quarter-round crest requires more concrete, compared to the half-round one.

In another work, Melton [126] examined a strategy for hybridizing two heuristic algorithms, PSO and DE, to better address the stagnation problem when solving slew-maneuver time-optimal problems having constraints in solution path. The stagnation was handled very well by the combined PSO-DE method, with a reduction of 40% in the computational time compared to just applying DE. In [34], a Dynamic Multi-Swarm Differential Learning Particle Swarm Optimizer (DMSDL-PSO) was proposed. In this study, the DE operators were incorporated into each one of the DMSDL-PSO’s sub-swarms to form a novel method. DMSDL-PSO has a good capability of exploitation and exploration by employing a local searcher like Quasi-Newton method to enhance the exploitation capability, making, at the same time, use of the capability of exploration existing in the differential mutation. On the other hand, the optimization performance of DE was improved by developing a novel self-adaptive mutation DE algorithm based on PSO (DEPSO) [194]. DEPSO can significantly utilize the fast convergence capability of PSO and the strong global exploration capability of an improved DE mutation strategy. As a result, the diversity of the swarm was managed well throughout the evolution, resulting in a higher convergence speed.

In [12], Aydilek proposed a Hybrid algorithm combining both FA and PSO (HFPSO). HFPSO checks the global historically best fitness values in order to properly determine the proper start of the local search. The high accuracy of convergence and runtime was statistically approved by evaluating the method using expensive benchmarks from the high-dimensional CEC’15 and CEC’17 functions.

Typically, a strong classifier can be created based on the strengths of penalization and, say, SVM, proving the effectiveness of Penalized Support Vector Machine (PSVM). In [6], Al-Thanoon et al. proposed a new hybrid PSO and FA to find the tuned parameters of PSVM, showing a high ability to escape from being trapped into the local optima trap.

In [223], a hybrid method of PSO and GA was proposed by Zhu et al. to optimize an antenna array deployment for locating the sources of Partial Discharge (PD) in an entire substation. First, an algorithm called Direction-of-Arrival (DOA) estimation was presented for testing its applicability to arbitrary array configurations. In order to minimize the objective functions, the hybrid PSO-GA algorithm was applied to optimize the array deployments for the localization of both DOA and coordinates.

Genetic Learning PSO (GL-PSO) breeds eminent exemplars to steer the motion of particles for the purpose of improving the performance of PSO. However, the performance of GL-PSO is not satisfactory on complex optimization problems, in which a global topology is depicted for exemplary generation and efficient diversity cannot be retained to boost exploration. For the betterment of the adaptability and performance of GL-PSO, Lin et al. [106] modified two versions of the basic GL-PSO algorithm. In exemplar generation, a ring topology is depicted to boost diversity and so exploration, while the algorithm’s adaptability is improved by adopting a Global Learning Component (GLC) with control parameters which are linearly adjusted.

Nenavath et al. [135] proposed a novel hybrid algorithm of Sine Cosine Algorithm (SCA) and PSO (SCA-PSO) to overcome the SCA premature convergence at local minima for the two major goals of object tracking and solving optimization problems. Two parameters of PSO, \({\mathbf {p}}_{best}\) and \({\mathbf {g}}_{best}\), were embedded into the conventional SCA to lead the process of pursuing the possible candidate solutions, so that the \({\mathbf {p}}_{best}\) in SCA is then used to initialize PSO swarm to further exploit the feasible search spaces.

PSO usually experiences premature convergence, so it is easily to be caught into a local optimum solution. Also, it is ineffective in balancing exploration and exploitation, especially in complex multipeak search functions. To overcome these shortcomings, a Hybrid Particle Swarm Optimizer with Sine Cosine Acceleration Coefficients (H-PSO-SCAC) was proposed [32]. First, Sine Cosine Acceleration Coefficients (SCAC) were presented to efficiently control the convergence to the global optimum solution by guiding the local search. Second, the swarm is initialized by adopting opposition-based learning method. Additionally, the inertia weight is adjusted by utilizing a sine chaotic map. Finally, a modified formula for updating position was proposed.

The SCA is mainly stigmatized by its limitation to exploit only emerging regions, depending on the parameter settings. However, the SCA is well capable of exploration. Taking advantage of the PSO capability of strong exploitation, Issa et al. [73] developed a new enhanced version of the SCA by hybridizing it with PSO. Two layers have formed the resulting technique: In the bottom layer, the search agents of SCA explore the search space, and in the top layer, the region around the best result found by the layer at bottom is exploited. Hence, there is a balance between exploitation and exploration in the proposed technique, which maintains fast convergence and improves the solution quality.

In another research, Sanchez et al. [159] used a column generation framework to solve the Binary Cutting Stock (BCS) problem by developing a benchmark of different approaches. Master- and sub-problem are the main components of this framework. Classical integer linear programming was used to solve the master problem, while meta-heuristic algorithms (PSO, SA, and GA) were used to solve the sub-problem. The aim of this benchmark analysis was to compare the results of the hybrid meta-heuristics with an exact approach.

In [68], Huang et al. presented a methodology to utilize various loading-unloading curves to extract the mechanical properties of its bulk materials using an Instrumented Indentation Test (IIT). Simulated Annealing PSO (SAPSO) coupled with finite iterative element simulation were used to obtain the mechanical properties, including strain-hardening rate (K) and strain-hardening exponent (n), from the IIT data.

Ibrahim et al. [70] proposed a hybrid optimization approach for solving the feature selection problem. The Salp Swarm Algorithm (SSA) was combined with the PSO as SSAPSO, in which the performance of the exploitation and exploration stages in PSO was significantly amended. Based on 15 benchmark functions alongside different UCI datasets, the original datasets were refined by removing the confusing or redundant features while yielding or keeping a better efficiency, demonstrating the improvement in the SSAPSO with respect to the prediction accuracy without affecting the computational cost.

Laskar et al. [98] proposed a new hybrid algorithm named as the Hybrid Whale PSO (HWPSO) algorithm. Since Whale Optimization Algorithm (WOA) is known for its very good capability of exploration, it is hybridized with PSO in a novel way to overwhelm the limitations related to a PSO phase (i.e., stagnation impact). The HWPSO has employed two novel techniques during hybridization: “Forced” whale in the exploration phase so that the WOA can guide PSO to better avoid getting stuck into local optima, and “Capping” phenomenon in the exploitation phase for restricting the search mechanism of WOA in order to converge to the global optimum value more fast.

A new form of multi-swarm BBPSO, in which its updating distributions are reoriented by adaptively learning the optimal alignments, is presented by Vafashoar and Meybodi [183]. The idea is that, along appropriate directions, the prospect of generating new particles is maximized. Due to the lack of a priori knowledge of these alignment directions, the proposed method adaptively learns suitable alignments by employing a learning mechanism. Cellular learning automata were adopted to build this learning mechanism. For each particle in the presented method, multiple alignment strategies are developed. Moreover, these strategies are adjusted so that, during the search process, the particles are attracted toward the most promising regions based on the cellular learning automata.

Durán-Rosal et al. [43] proposed novel approaches based on time series segmentation. The proposed methods include the PSO algorithm adaptation to this problem, as well as more advanced variants of PSO (e.g., BBPSO and its exploitation variant (BBePSO)). Furthermore, a new algorithm, called Dynamic exploitation BBPSO (DBBePSO), was derived, in which the importance of the cognitive and social components are updated throughout the successive generations. A final local search step is considered to further improve the solutions obtained from these algorithms based on the incorporation of two popular traditional segmentation algorithms (Top-Down and Bottom-Up).

In [134], Nagra et al. proposed an enhanced Self-Inertia Weight Adaptive PSO algorithm with a gradient-based Local Search strategy (SIW-APSO-LS) to solve the problem of premature convergence in PSO. This proposed algorithm takes the advantages of the exploration ability of the adaptive PSO with enhanced inertia weight, and the exploitation of the gradient-based local search strategy.

In [30], Cao et al. proposed a Comprehensive Learning Particle Swarm Optimizer (CLPSO) embedded with a local search utilizing the CLPSO’s capability of global search and the fast convergence capability of local search to achieve higher optimization performance. This work takes advantage of the introduced so called quasi-entropy index for addressing its main issue (specifically, when to start the local search) by developing an adaptive local search starting scheme.

In most of the above-discussed studies, PSO was implemented without coherence with other meta-heuristics. For instance, a good initial solution can be produced by utilizing DE or GA as a starting point, and the PSO can take care of the rest of the search. In some cases, an initial solution can be produced by using PSO as a starting point, and the search can be conducted by others. Furthermore, the PSO, as a supplementary tool, may be hybridized with the deterministic local search to boost the exploitation capability and produce high-quality solutions. Additionally, the search speed may be elevated by the PSO.

5.1.2 Artificial Neural Networks (ANNs)

Artificial Neural Networks (ANNs) can be, in a word, summarized as learning via training [187]. To achieve that, a series of input and output vectors is adopted to handle a set of data that needs to be trained. At the training time, the ANN is applied to the training data iterative, so that the desired input-output mapping emerges by balancing weights of the network several times in a while. After the training stage completes, a foreseeing model is ready and the corresponding output vector can be then created from input ones unrelated to the training pairs. An appropriate set of weights and transferring functions should be carefully chosen (via, say, meta-heuristic optimization) to ensure that the ANNs can present a quantifiable smooth function that would properly link future input and output vectors. Various components, including prediction and system control and modelling, are inferred to express the broadly connected neural systems.

In this regard, Moayedi et al. [131] introduced a PSO-optimized ANN model to solve the prediction problem of Landslide Susceptibility Mapping (LSM). The focus of this study was the prediction of landslide hazardous susceptibility mapping by applying a hybrid model of PSO and ANN (PSO-ANN). Two statistical performance metrics, Root Mean-Squared Error (RMSE) and coefficient of determination (\(R^2\)), were used to assess the predicted results from both the original ANN and the PSO-ANN. Comparing the two models, the PSO-ANN model was observed to be more reliable in LSM estimation than the ANN.

In another work, Junior and Yen [84] proposed a novel algorithm based on PSO and Convolutional Neural Network (CNN), namely, PSO-CNN. In comparison with other evolutionary approaches, the proposed algorithm is capable of fast convergence. In an application to image classification, it was able to automatically find deep meaningful CNN architectures. To allow the use of PSO optimization with CNN, a novel strategy for direct encoding, as well as a velocity operator were built.

5.1.3 Support Vector Machine (SVM)

Support Vector Machine (SVM) [56] is an intelligence algorithm lying under supervised machine learning [2]. In this algorithm, the data is analyzed and particular visible or hidden patterns are quested with the major aim to perform two main tasks of classification and regression. SVM is known for its potential to perform non-linear classification and linear separation of the data, to build another dimensional space from existing training datasets by investigating the respective categories. The separation process is working by increasing the chasm between too close points of various categories (i.e., support vectors) to better explore the optimal separating hyper plane. It should also point out that SVM can perform other tasks like linear classification [41].

In this framework, Hoang et al. [61] proposed a novel Differential PSO-based SVM (DPSO-SVM) classifier to monitor the conditions of a surge arrester. In the proposed method, the parameters of SVM classifiers were optimized by investigating the DPSO-SVM technique configuration that gives the best results. Input vectors composed of three extracted features are used to assess five arrester conditions, including degradation (D), tracking (T), pre-fault (A), normal (N), and abnormal (U). Meanwhile, a DPSO-based ANN classifier was also involved, to carry out a comparative study on fault diagnosis.

In another work, Tharwat and Hassanien [179] optimized the SVM parameters by employing quantum-behaved PSO (QPSO) for reducing the classification error. The proposed model, called QPSO-SVM, was evaluated by adopting 7 traditional classification datasets from the UCI data repository. The experimental results revealed the capability of the proposed model to obtain the best values of the parameters of SVM. Moreover, the results showed lower classification error rates than those obtained by the standard versions of PSO and GA.

5.1.4 Quantum

40 years ago, quantum computer [20] was invented, and in the late 1980s, the quantum computer formal definition was given. Due to the potential of the quantum computer in various special problems [85], this field has witnessed many great efforts. Various popular algorithms are developed, and the most well-known one of them is Shor’s quantum factoring algorithm [185]. In 2004, Sun et al. [170] firstly introduced the Quantum-behaved PSO (QPSO) for improving the convergence rate of the native PSO. In quantum space, particles often have the guarantee to reach the real global optimum via searching throughout the full solution space. Later in recent decades, QPSO has drawn great attention from many scholars. In what follows, some of them are presented.

In [81], Jmal et al. proposed a QPSO method for solving the K-Traveling Repairman Problem (K-TRP). The proposed approach incorporates a K-TRP-specific repair operator to ensure that a feasible solution space will be always there to guide the search process and that the quality of solutions is significantly improved as much as possible.

In [147], an optimal path planning was proposed for free-floating two-wheel pendulum robot system based on its self-balance. First, the corner trajectory of this pendulum robot is parametrized by QPSO. The native attitude and the control precision of the robot’s terminal attitude and position are utilized to formulate the objective function. The addressed problem of optimal path planning was considered as a non-linear optimization problem that is solved using the QPSO algorithm to achieve the demanded goal.

Motivated by the characteristics of PSO and quantum theory, Xu et al. [199] proposed a new method, namely, QPSO. The QPSO used fractional calculus to enhance its global search ability. The discrete expression of the Grünwald-Letnikov definition, one of the most popular differential fractional definitions, was incorporated for better updating of the particles’ positions in QPSO.

5.1.5 Rough Set

The theory of rough set [142] is an approach devised to deal with uncertainty and ambiguity. The philosophy of this approach was built on the fact that every object in the universe has its own characteristics. The rough set theory is a mathematical tool utilized to extract knowledge from information obtained from uncertain data [105]. In feature selection, rough set analysis is utilized because it only requires the supplied basic data without the need for any supplementary information. Also, it is characterized by the suitability for exploring the qualitative and quantitative properties of data.

Fan et al. [51] proposed an algorithm called RoughPSO for solving the convergence to a local optimum in PSO, depending on the rough set theory. The RoughPSO uses upper- and lower-approximation rough sets to get the membership values. Then, these values were employed to refine the velocity and position for each particle.

In another work, Bhattacharya et al. [23] proposed a technique of Feature Selection based on an impromptu PSO algorithm and Rough Set (PSORS-FS), to select the most relevant features for detecting permission-based Android malwares. This work mainly contributed to the recommendation of a new method of random key encoding that converts the conventional PSO algorithm in discrete space. It also reduced the issues related to the sigmoid function in binary PSO, as well as the particles’ maximum velocity.

5.2 Improved PSO

This section is competent of the improvements in PSO based on some third-party methods as learning strategy, fuzzy logic, mutation, Lévy flight, opposition-based learning, and surrogate.

5.2.1 Learning Strategy

In reality, increasing complexity in many optimization problems calls for further research on the exploration and improvement of diverse optimization algorithms. It has been proved that the PSO algorithm is an efficient tool to solve different types of optimization tasks. However, for the base PSO, the main aim of the updating strategy is to learn the global optimum, and it always experiences too fast convergence in addition to poor performance on various complex optimization tasks, principally for multimodal problems. The mechanism of learning in classical PSO is simple and easy to develop, but some likely issues, such as the phenomena of “two steps forward, one step back” and “oscillation”, may disrupt it. Therefore, PSO researchers have managed to design new efficacious learning strategies for avoiding the two phenomena and improving the overall search performance. Xu et al. [198] presented a Dimensional Learning Strategy (DLS) following the best personal experience of each particle to eventually discover and integrate the propitious information extracted from the optimal solution of swarm. Thereafter, different learning strategies are used to provide a Two-Swarm Learning PSO algorithm (TSLPSO). One of the two sub-swarms used the DLS to create learning exemplars for guiding the particles’ local search, while the other sub-swarm used the comprehensive learning strategy to create learning exemplars for managing the global search of particles.

Wang and Liu [193] proposed a Heterogeneous Comprehensive Learning PSO (HCLPSO) algorithm to attain 3D spatial trajectory tracking to realize a new saturated approach to control a quadrotor. First, the quadrotor model was divided into: an inner position control loop, inside cascaded control structure, and an outer attitude control loop. Second, the saturated control is applied to limit the quadrotor’s thrust force in the outer attitude control loop. Next, the parameter adjustment difficulty for the quadrotor was alleviated by employing the HCLPSO algorithm to optimize the control parameters.

5.2.2 Fuzzy Logic (FL)

Since the performance of PSO is strongly dependent on the selection of its settings (i.e., minimum and maximum velocity, social and cognitive factors, and inertia weight), Fuzzy Logic (FL) can be first exploited to determine the best values of these parameters. So far, the PSO implementations based on FL aim at calculating distinct settings for the swarm as a whole. Nobile et al. [138] proposed a new algorithm called Fuzzy Self-Tuning PSO (FST-PSO) which, for every particle, independently calculates the minimum and maximum velocity, social and cognitive factors, and inertia by exploiting FL, thereby creating a completely setting-free variant of PSO. The strength and novelty of FST-PSO was derived from the fact that there is no experience required to formulate PSO, since the optimization process automatically adaptively adjusts the behavior of each particle.

In [114], López et al. proposed a Fuzzy Logic Controller (FLC) modified by PSO, namely, Fuzzy-PSO to increment the lifetime of power electronics with a faster response of drive’s speed in a brushless DC electric motor. Furthermore, the reference temperature and the desired speed are proposed as parameters to formulate an objective function.

5.2.3 Mutation

Salajegheh and Salajegheh [157] combined PSO with gradient directions of first and second order, thereby achieving a great increasing in the approach robustness. The literature was thoroughly searched to choose an appropriate set of design problems, and the results of the standard PSO and the counterparts are compared with each other. In the proposed approach, the overall optimization cost was trivial as a result of decreasing the number of initial particles.

A modified variant, namely, Repository and Mutation based PSO (RMPSO) was proposed in [75]. In RMPSO, global and personal best solutions with the same fitness values are stored in two maintained extra repositories. And, another proposed Enhanced Leader PSO (ELPSO) indicated that the swarm optimum could be improved by applying five successive mutation strategies.

5.2.4 Lévy Flight (LF)

Lévy Flight (LF) is a particular category of random walk in which a heavy power tails law is applied for the distribution of step lengths. A global search is done or conducted by an algorithm with the occasional aid of larger steps. A better trade-off between the two capabilities of exploration and exploitation in an algorithm could be gained by applying the LF trajectory [47], and the ability to avoid local optima gives plus points.

In [213], Zhang et al. established an optimization model for the task scheduling model in the Multiple-Input and Multiple-Output (MIMO) radar, and proposed a hybrid Discrete PSO (DPSO) algorithm with LF for solving the problem. The internal structure of tasks, the features of the MIMO radar task scheduling, and the three main principles of task scheduling were all considered in the optimization model. In another work, Tang et al. [175] employed the signal level of Automatic Identification System (AIS) to build a new method for estimating the atmospheric profile refractivity, specifically the parameters of surface-based duct in the maritime environment. Due to the complexity and non-linearity of this optimization problem, LF with QPSO algorithm (LF-QPSO) was proposed to find the optimal solution.

In [100], Li et al. developed a novel Unsupervised LF with PSO method (ULPSO) for image classification, giving a good exploration-exploitation balance. ULPSO depends on a new searching mechanism that targets the worst particle in the swarm at each iteration to update its position with LF.

5.2.5 Opposition-Based Learning (OBL)

Tizhoosh [181] originally introduced the basic idea of Opposition-Based Learning (OBL). Based on the concept of OBL, typically, the fitness values of current agents and their opposites are evaluated [119]. Then, the suggested algorithm selects N agents with the highest/lowest fitness values.

In [86], Kang et al. proposed a hybrid PSO algorithm to improve the PSO performance by maximizing PSO diversity introduced by OBL. Probabilistic OBL was adapted for a swarm by the proposed hybrid algorithms. Unlike other fusions of PSO and OBL, the fitness of the entire swarm was improved by selecting the fittest particles from the current swarm and its opposite one.

In [189], Wang et al. addressed the learning strategy impact on a scheduling problem with two-stage assembly flow shop. Three different machines were used to, in turn, process several different workpieces. The total completion time was minimized by developing a branch-and-bound technique combined with a lower bound procedure and several developed dominance rules to obtain the optimal scheduling solution. Besides, high-quality feasible solutions were obtained by devising a new Dominance Rule-based Opposition-based PSO algorithm (DR-OPSO).

5.2.6 Surrogate

Surrogate-assisted optimization was established to handle expensive and complex problems in real-world applications. The methodology of this type of optimization depends on the maximal exhaustion of the available information for minimizing the number of expensive evaluations required for a given objective function, thereby reducing the related costs, resources, and time [169].

Related to this, Yu et al. [207] proposed a hierarchical surrogate-assisted particle swarm optimizer comprising a Social Learning-based PSO algorithm (SL-PSO) and the standard PSO algorithm itself with the goal of solving high-dimensional problems. The cooperation between SL-PSO and standard PSO was proposed so that the search space can be thoroughly exploited and explored, and the surrogate model global and local search performance is simultaneously improved.

In [203], Yang et al. proposed an Improved Surrogate-Assisted PSO (ISAPSO) algorithm, in which global and local surrogates are integrated into a hybrid PSO. The global surrogate uses an efficient global optimization algorithm to work as a global searcher for speeding up the PSO global search process, and reduces the computational burden by predicting the fitness values. Meanwhile, the local surrogate explores the expected optimum solution so that it can later locally search through the neighborhood of this solution.

5.3 Variants of PSO

Different variants of the PSO algorithm are addressed in this section.

5.3.1 Binary

Continuous meta-heuristics cannot handle a variety of optimization problems. Therefore, combinatorial optimization has emerged, which includes a type of optimization called binary optimization. The {0, 1} set elements are distinct decision variables in binary optimization problems. Thus, the binary optimization problem can have 0 and 1 as potential values for each decision variable. In other words, the decision variables can be digitally expressed as 0 or 1 with respect to the binary optimization problem at hand.

Sameer et al. [158] proposed a new Modified Binary PSO with a Kernel Fuzzy Clustering Method (MBPSO-KFCM) for selecting relevant features and determining the number of clusters in fuzzy data clustering. To enhance the quality of cluster analysis, Gustafson-Kessel (GK) algorithm was established and improved. The datasets Taiwanese, German, and Australian credit were used to test the performance of the three proposed algorithms. The results showed that fuzzy partition (classification) is robust based on the fuzzy Rand validity metrics, which provides an opportunity to reduce the risk associated with loans.

In [89], Karbassi Yazdi et al. proposed a Binary PSO variant (BPSO) for optimally solving the problem of ship routing and scheduling in Liquefied Natural Gas (LNG) extraction, transportation, and regasification.

5.3.2 Chaotic

Generally, chaotic, adapted from the term “chaos”, formally refers to such a property that describes the unpredictable behavior of a complicated system. In chaotic systems, a function is used to “map” or associate algorithm chaotic behavior with some parameters, using two typical chaotic maps: logistic and sine. Chaos has distinctive properties of non-repeatability and periodicity. So, it has the ability to do general searches at higher speeds compared to the probabilistic stochastic searches [72].

In [200], Xu et al. proposed a new Chaotic Search PSO algorithm (CS-PSO) which enhances the PSO algorithm with the CS method for solving combinatorial optimization problems. In the initialization phase, the particles were positioned depending on the advance knowledge of the combinatorial optimization problems. Furthermore, in the chaos perturbing phase, the positions and velocities of particles are perturbed by introducing a brand-new range of rules for satisfying the adaptability and the capability of the ideal global search with the major aim to effectively avoid the fruitless early convergence that haunts the standard PSO algorithm.

In another work, Wang et al. [188] presented an efficient multi-objective optimization approach that combines an adaptive chaotic PSO and the IsoGeometric Analysis (IGA) to optimize the Ceramic Volume Fraction (CVF) distribution under eigenfrequencies in Functionally Graded Plates (FGPs). And, the B-spline basis function was used to represent the CVF distribution.

In [209], Yue et al. proposed a hybrid PSO algorithm based on the chaos theory for solving the mobile location estimation problem. In the proposed location estimation approach, it was crucial to consider the influence of measurement error arising from the nonlinear optimization problem and Angle Of Arrival (AOA)/Time Difference Of Arrival (TDOA) hybrid location method. The proposed algorithm recognized the AOA/TDOA hybrid location algorithm to track the object to significantly enhance the locating accuracy and the performance of the estimation process in general.

5.3.3 Multi-objective

Multi-Objective Problems (MOPs) [108] is to be introduced in this section. Minimizing or maximizing multiple conflicting objective functions are two main tasks often considered in MOPs [148, 221]. Unlike single objective optimization ones, MOPs involve multiple contradictory objective functions. It is not easy to simultaneously optimize many functions. Hence, for obtaining an optimal solution set, there should be balance between the objective functions. Some constraints and at least two objectives must be included in a multi-objective optimization problem. In a solution space, the objectives and constraints are satisfied through finding mutually acceptable solutions.

In [214], Zhang et al. proposed a multi-objective particle swarm optimizer based on a competitive mechanism, in which the current swarm performs, at each iteration, pairwise competitions which in turn are used to update the position particles. On the other hand, a bi-objective problem was formulated for the hand posture estimation problem [160]. To solve this problem, the concepts of Evolutionary Population Dynamics (EPD) were employed for developing an improved version of MOPSO. This problem has different types of parameters. Therefore, some of the parameters were significantly calibrated by the MOPSO algorithm.

In [219], Zheng et al. presented an adaptive neighborhood function for developing a Multi-Objective Cellular PSO (MOCPSO). Three objective functions were taken for drilling a wellbore trajectory less costly, more quickly, and safely, compared to other potential trajectories. Then, a set of Pareto optimal solutions is gained by applying the MOCPSO algorithm to the three objective functions for designing a cheaper and safer wellbore trajectory.

Adhikari and Srirama [5] developed a new Container-based Energy-Efficient Scheduling method (CEES) that responds effectively to different types of fast submitted Internet of Things (IoT) and non-IoT tasks. A Multi-Objective Accelerated PSO (MOAPSO) technique was used in the proposed method to ensure a minimum delay by finding the most appropriate container for executing each task. To better utilize cloud resources, cloud environment involves other important missions of resource scheduling, which can be explored in [64].

In accordance with the SR process adopted in this study, this section is to provide a technical review of the designated PSO applications in the actual literature. Figure 5 demonstrates a comprehensive taxonomy of the PSO applications in different domains, including health-care, environmental, industrial, commercial, smart city, and general aspects. Each category of PSO applications is likely to face some issues that should be highlighted to come up with thriving solutions, enabling further efficient and viable PSO implementation in future real-world applications. Consequently, studies focusing on some pivotal issues are reviewed to prop PSO applications in a particular context associated with these issues. For example, in environmental applications, the main contexts, such as economic emission dispatch, parameter identification of PhotoVoltaics (PV), pollution forecasting, segmentation and classification of plants, flood control and routing, water quality monitoring, and many other issues, are floated in different aspects of environmental PSO applications. Thus, this paper presents a taxonomy based on diverse categories of PSO applications in selected research studies in which special subjunctives are addressed and discussed. Considering the concerns and challenges in various types of PSO applications, I firstly addressed different categories in the PSO applications and then reported the main subjects which have been paid special attention in each category. PSO applications have some general concerns, so a division in the taxonomy, namely “general aspects”, was adopted to refer to the studies that introduced a proposal to cope with a particular challenge in any general type of PSO applications. In other words, a new conceptual approach is introduced based on the studies shown as being of general aspects, thus promoting the development of any type of PSO application.

The taxonomy of PSO applications

Full size image

The upcoming subsections present diverse approaches in PSO applications. In addition, various researches will be compared from several sides, such as key subject, case studies, strengths, shortcomings, and special outputs.

6.1 Health-Care Applications

Table 3 illustrates a summary of some seminal researches and important information to assess health-care approaches in PSO applications. Some of the major contexts in the health-care domain are based on intelligent diagnosis, disease detection by medical robots, medical image segmentation, and disease classification.

6.2 Environmental Applications

Table 4 illustrates a summary of the recent researches and important information to assess the environmental approaches in PSO applications. The approaches include case studies, such as wild vegetation environmental monitoring, agriculture environmental monitoring, flood control and routing, water quality monitoring, and pollutant concentration monitoring systems.

6.3 Industrial Applications

Table 5 illustrates a summary of the recent researches and important information to assess the industrial approaches in PSO applications. Some of the major contexts in the industrial domain include economic dispatch problem in power systems, optimizing Phasor Measurement Units (PMUs) placement, allocating daily electrical loads, deploying WSNs, predicting the defection of products, and optimizing the design and operation of microgrids.

6.4 Commercial Applications

Table 6 illustrates a summary of some seminal researches and important information to assess the commercial approaches in PSO applications. Some of the major contexts in the commercial domain include prediction of cost and price, risk assessment, and profit calculation.

6.5 Smart City Applications

Table 7 illustrates a summary of some seminal researches and important information to assess the smart city approaches in PSO applications. The approaches include case studies, such as smart city, smart home, appliance scheduling, and vehicular monitoring systems.

6.6 General Aspects of PSO Applications

Table 8 describes a summary of some seminal researches and important information to assess the general approaches in PSO applications. Some of the major contexts in the general domain include service allocation, image segmentation, scheduling, prediction, and security management.

Similar to many other meta-heuristic algorithms, PSO has distinctive advantages, as well as some unavoidable shortcomings. Although there is no evidence of convergence for this algorithm, the results presented in this study indicate its competitiveness over, for example, EAs in terms of convergence rates and accuracy in many cases. Table 9 lists some of the impacting strengths and weaknesses of PSO.

Previous sections discussed the review process of the selected studies regarding PSO methods and applications. In turn, this section statistically analyzes those methods and application areas. Furthermore, the proposed AQs proposed in Sect. 4 are called to present some analytical reports as follows:

• AQ1: Into which domains can PSO methods and applications be classified?

According to the taxonomy exhibited in Sect. 5, Fig. 6 depicts a comparison of the percentages of PSO methods as of the date of this study. I considered three PSO methods that span hybridization, improvement, and variants. As shown in Fig. 7, PSO variants have the largest share of PSO methods in the literature at 42%. Of course, hybridization techniques have 32%, and improved PSO approaches have 26% usage of the PSO methods.

Percentage of the presented PSO methods

Full size image

Total percentage of the presented PSO methods

Full size image

Similarly, according to the taxonomy exhibited in Sect. 6, Fig. 8 depicts a comparison of the portions of PSO applications as of the date of this study. Six PSO application domains, including health-care, environmental, industrial, commercial, smart city, and general aspects applications, are considered. If we look closely, the general approach has the largest percentage of application areas in the literature with a usage ratio of 27%. Of course, industrial applications have 20%, environmental applications have 17%, smart city applications have 16%, health-care applications have 11%, and commercial applications have 9% usage out of the overall PSO applications domain.

Percentage of the presented PSO applications

Full size image

• AQ2: Which main contexts can be considered for PSO methods and applications?

Within the framework of this study, the main contexts of PSO methods and applications are shown in Fig. 9. It has been observed that the electrical engineering field received the most attention from scholars and practitioners with 13 studies, while prediction, image processing, and mechanical engineering approaches were 6 studies.

Percentage of the main contexts for PSO methods and applications

Full size image

• AQ3: What assessment environments are used for evaluating the PSO methods and applications?

According to Fig. 10, it has been observed that 47% of the research studies apply simulation tools to evaluate the presented case studies in the PSO community. In addition, it has been observed that 33% of the research papers have implemented approaches to develop PSO methods and applications. Moreover, 11% of the researches have not presented any simulation or implementation for the indicated methods and application domain. Finally, 9% of the existing studies use datasets to apply analysis approaches (e.g., testing and prediction) for assessing the involved case studies.

Percentage of the PSO assessment environments offered in the literature

Full size image

While PSO has been effectively applied in versatile domains, challenges are required to be addressed and drawn as future research directions. In the past few years, PSO has gained an increasing interest from researchers and has been utilized in different areas of applications. However, critical problems and issues remain. Thus, more research efforts should be done by scholars and researchers to conquer the challenges and problems that may hinder the future application of PSO. Moreover, further inspirations and more effective techniques should be pursued to develop novel PSO approaches. For example, the research community needs to address new methodologies for complex problems. Based on the literature review, I discuss below different open issues and related topics for potential future research.

I should point out that the assortment of PSO methods and applications mentioned in this study applied the SR process so that the challenges discussed in the following two subsections are presented as the AQ4:

• AQ4: What are generally the PSO open perspectives and future research directions that can be drawn from the work presented in the paper?

8.1 Open Issues

Premature convergence When applying the traditional PSO algorithm, if the search gets closer to the local optimal solution through random initial conditions, the optimal solution for a single particle (individual particle optimum) and the optimal solution for all particles (the group/global optimum) would both converge towards the local optimal solution, and a deceptive optimization result will therefore be obtained. Thus, it is not possible to guarantee the capability of finding the global optimal solution. As a result, the fast convergence capability will be ineffective. Over the past few years, the issue of stagnation (premature convergence) has been a subject of research in the PSO community [197, 208]. A roadmap for future research in this regard can be created based on some topics, including particles’ stability analysis [26], redistributing mechanism [146], and random sampling of control parameters [171].

Convergence speed controller Despite the typical fast convergence of PSO, it may be trapped into a local optimum due to premature converge. Therefore, a convergence speed controller was devised for the ultimate goal of solving this problem. Two adaptive approaches were proposed to adjust the convergence speed. First, when the particle prematurely converges, the convergence speed is slowed down. Second, when the particle cannot update its best solution in the present time, the convergence speed of PSO is accelerated. Thus, a key residual challenge is developing an effective convergence speed controller [110]. In the same context, for improving the convergence speed of PSO-based algorithms, the social and cognitive components of the velocity update formula are modified based on the fitness of each particle [125]. In addition, the convergence speed is adaptively controlled to empower the PSO algorithm for solving large-scale numerical optimization problems [66]. Hence, designing an appropriate convergence speed controller framework for PSO has become a major challenge in this topic.

High-dimensional search space Because of the curse of dimensionality problem, the effectiveness of PSO application to classification of high-dimensional data has recently gained a major concern [9]. In a recent study, the effectiveness of PSO approach for feature selection has been demonstrated. However, due to the large search space, the challenge sill exists to apply PSO-based feature selection to high-dimensional datasets with tens of thousands of features [182]. In addition, the PSO algorithm can be adapted for the problem of high-dimensional feature selection, providing thereby an efficient technique for achieving similar or even better classification accuracy by not using all features but instead selecting only a small set of relevant features from a wide range of features [58]. The key challenges in this area include novel approaches like Monte Carlo methods [22] that simultaneously minimize the number of chosen features and maximize the classification accuracy in PSO applications.

Memory requirement All living systems have memory as an essential feature. According to the evolutionism perspective, it is worthwhile examining the potential helpful role of historical memory in the process of EAs. That is mainly because the explicit or implicit historical memory in PSO improves the search process by storing promising solutions and reusing them in later stages [101]. Furthermore, a new inertia weight can be generated by using historical memory through a parameter adaptation mechanism. Therefore, improving PSO using historical memory is a pressing challenge [60, 103]. And how to set the memory size adaptively can be also considered as an interesting topic to be addressed in the future [117].

Parameter & topology selection In PSO-based algorithms, the best performance can be achieved by elaborately determining the control parameters. However, choosing these parameters cannot be adequately guided. To address the job of parameter selection, future efforts in this topic should include choosing the best parameters based on simulations [38], parametric analysis in a computational environment with limited resources [162], and hyper-parameter selection based on heuristics [115]. On the other hand, the performance of PSO in engineering applications is significantly affected by topology selection, and each problem has its appropriate optimal topology. However, this issue still lacks much study. As a future direction, topology selection for PSO can be better guided by taking into account factors affecting the optimality of algorithmic parameters (i.e., the topological degree and the number of particles) with the aim of selecting a proper class of deterministic regular topologies [112]. Other challenges including, studying and evaluating the performance of tree topology [155] and suggesting different topologies for the PSO optimizer in order to reach, for example, an optimal Proportional-Integral-Derivative (PID) controller design [3], are also pressing issues which calls for further research and audit.

8.2 Future Perspectives

PSO was practically applied for the first time to the domain of neural network training, which was indicated by the algorithm itself [93]. Many more areas of PSO methods and application have been explored ever since, including signal processing, power systems, combinatorial optimization, design, data mining, control, telecommunications, and many others. To date, applications of PSO algorithms are reported in hundreds of publications [46, 59]. Although the main use of PSO is to solve unconstrained, single-objective optimization problems, a development has been made to PSO algorithms for solving constrained problems, multi-objective optimization problems, problems with dynamically changing landscapes, discrete problems, and for finding multiple solutions as well [16]. However, several PSO methods and applications can be sophisticated in some areas as future research directions in this topic as follows:

Nuero Fuzzy Network (NFN) Nuero Fuzzy Network (NFN) is an intelligent method for system identification, modelling, prediction, and control. In NFN, gradient-based algorithms are generally used for training. However, these algorithms have some disadvantages concerning, for example, getting stuck at local minima, which needs for complex gradient computations to be conquered. Accordingly, Karakuzu et al. [88] used improved PSO to introduce the first embedded high-speed, low-cost implementation of NFN hardware through online training. It has been observed that that the effectiveness of the proposed NFN implementation is similar to other approaches in the literature, thereby generating a novel idea for future research. In a different approach to NFNs, Vijay and Jena [186] utilized the PSO to minimize the quadratic performance indices for obtaining optimal sliding control parameters and PID parameters. In this study, Lyapunov stability theorem was used to improve the stability of the system by integrating sliding mode control into artificial neuro fuzzy inference system. Thus, it can be concluded that NFN can be applied to the robot manipulator for real-time control, where good control performance can be obtained by adaptively changing sliding control parameters under different input disturbances.

Color image processing In this regard, Gaussian PSO (GPSO) was used to develop a color image quantization algorithm [15]. In the proposed algorithm, each particle in the swarm is randomly initialized with k centroids (i.e. color triplets). The cluster with the closest centroid to a pixel in the image is recognized to assign this pixel to that cluster. The centroids obtained by K-means clustering are refined by applying the GPSO. Experimentation showed that using GPSO method for the color image quantization gives significantly better image quality than conventional PSO-based approaches. In anther work, a multi-level color image thresholding algorithm was developed on the bases of GLLA histogram and Tsallis-Havrda-Charvát entropy of degree \(\alpha\) [27]. Compared with the existing models, the proposed model achieved better performance results by applying the PSO algorithm. However, so far only a limited number of studies reported how to apply together the quantization and multi-level thresholding techniques to a color image.

Image registration Image registration is defined as taking several 2D images from various sources, such as Computer Assisted Tomography (CAT) and Magnetic Resonance Imaging (MRI) scans, and combining them into a 3D image. Recently, a hybrid approach for registering medical images has been developed by employing a PSO method [1] and an adjusted mutual information as a similarity index, or general meta-heuristics [184]. However, there are still trend applications for future research, including registering the images of a printed circuit board placed on a conveyor belt using an improved scale invariant feature transform, feature extraction technique combined with PSO [39]. Also, using PSO algorithm for remote sensing images registration less affected by the correction rate [196] is another major trend for future research.

Computational biology A long DNA chain first needs to be divided into subset fragments for determining its sequence. Therefore, combinatorial optimization researchers used the DNA Fragment Assembly method (DFA) to solve the NP-hard problem of reassembling the fragment. The DFA problem is solved by applying the overlap-layout-consensus model to maximize the overlapping score measurement using a memetic PSO algorithm based on two initialization operators, as well as the local search operator [67]. On 19 DNA fragment datasets, the results revealed that the PSO algorithm combining SA-based variable neighborhood local search and TS achieves the best overlap scores. Future works could address interesting issues, such as reducing the computational time by using DNA sequence compression, improving the initialization method of the algorithm, using an alternative search approach, and so on. In a similar biological application, some variants of the PSO algorithm have been utilized to extensively tackle the molecular docking problem and the problem of conformational search in protein-ligands [55, 136]. Ultimately, the literature has reported significant results. Thus, it is natural to solve biological problems by applying the PSO algorithm.

Recommender systems PSO was utilized as a tuning mechanism in a further area in which software tools are created to develop recommendations to entrepreneurs or even end users. Explicit feedback data (i.e., votes or ratings) are usually used to build most of the existing recommender models. However, real-life scenarios do not always contain explicit feedback data. For example, a hybrid music recommender system was suggested based on implicit feedback data by utilizing graph-based algorithms for making songs recommendations based on the user-s preferences and behavior [90]. Furthermore, PSO-based web-page recommendation system was developed on health-care multimedia data to track user navigation behavior by utilizing semantic web mining [121]. In the future work, these systems can be further extended to include other E-health care application, and/or social networking sites like twitter and Facebook.

In this literature, the exploration query on 3,632 papers published between 2017 and 2019 was executed to fulfill the proposed SR process. 2,140 papers related to PSO methods and applications were selected. Amongst them, I briefly discussed and analyzed only 84 papers to retain the proper length of the paper. According to AQ1, on the one hand, the PSO variants have the most percentage of the PSO methods in the literature by 42% of quota. Of course, hybridization techniques have 32%, and improved PSO techniques have 26% of portions of PSO methods. On the other hand, the general approach has the most percentage of the application domains in the literature by 27% of quota, emphasizing the broad application of PSO in diverse subject areas. Also, industrial applications, environmental applications, smart city applications, health-care applications, commercial applications have 20%, 17%, 16%, 11%, and 9% portions of all existing PSO applications, respectively. It has also been observed that electrical engineering approaches have the highest percentage, with 13 studies, and prediction, image processing, and mechanical engineering approaches are with 6 studies, based on AQ2. According to AQ3, 47% of the research studies applied simulation tools to evaluate the case studies presented in the PSO environment.

According to the adopted SR process, I must acknowledge that not all present studies may have been analyzed; non-peer reviewed, non-English book chapters, survey articles, and editorial papers were omitted. In this study, the findings of more than 120 authors and diverse studies were considered to perform a comprehensive research of the methods and application approaches in PSO. However, as the research ended in November 2019, it is not possible to guarantee that all relevant seminal works are covered herein, especially with the increasing number of studies in this field.

Finally, the SR method presented in this review focused primarily on PSO methods and applications. Through this study, the PSO methods and applications were comprehensively understood. In addition, open issues and future perspectives were considered. However, the PSO algorithm still needs further research in the following areas:

• Studying the effects of heterogeneity in PSO algorithms.

• Hybridizing PSO with novel optimization techniques, such as Salp Swarm Algorithm (SSA) [130], Whale Optimization Algorithm (WOA) [129], Lion Optimization Algorithm (LOA) [204], Elephant Herding Optimization (EHO) [191], and Jaya Algorithm (JA) [63, 153].

• Implementation of innovated smart city applications, such as smart metering, smart farming, smart logistics, and smart buildings.

• Further applications for addressing different issues in cloud computing.

• Addressing more complex/large-scale real-world problems, NP-hard problems, and discrete optimization problems to discover new limitations of PSO.