More Web Proxy on the site http://driver.im/

article

Training neural networks via simplified hybrid algorithm mixing Nelder---Mead and particle swarm optimization methods

Authors:

Jer-Guang Hsieh,

Jyh-Yeong Chang,

Chin-Teng LinAuthors Info & Claims

Soft Computing - A Fusion of Foundations, Methodologies and Applications, Volume 19, Issue 3

Pages 679 - 689

https://doi.org/10.1007/s00500-014-1292-y

Published: 01 March 2015 Publication History

Abstract

In this paper, a new and simplified hybrid algorithm mixing the simplex method of Nelder and Mead (NM) and particle swarm optimization algorithm (PSO), abbreviated as SNM-PSO, is proposed for the training of the parameters of the Artificial Neural Network (ANN). Our method differs from other hybrid PSO methods in that, $$n+1$$ n + 1 particles, where $$n$$ n is the dimension of the search space, are randomly selected (without sorting), at each iteration of the proposed algorithm for use as the initial vertices of the NM algorithm, and each such particle is replaced by the corresponding final vertex after executing the NM algorithm. All the particles are then updated using the standard PSO algorithm. Our proposed method is simpler than other similar hybrid PSO methods and places more emphasis on the exploration of the search space. Some simulation problems will be provided to compare the performances of the proposed method with PSO and other similar hybrid PSO methods in training an ANN. These simulations show that the proposed method outperforms the other compared methods.

References

[1]

Ahmadi MA, Ebadi M, Shokrollahi A, Majid SMJ (2013) Evolving artificial neural network and imperialist competitive algorithm for prediction oil flow rate of the reservoir. Appl Soft Comput 13(3):1085-1098.

Digital Library

[2]

Blake C, Merz C (1998) UCI Repository of Machine Learning Databases. http://www.ics.uci.edu/~mlearn/MLRepository.html

[3]

Bertsekas D (1999) Nonlinear Programming. Athena Scientific, Belmont.

[4]

Chen J, Tu X, Fu J (2005) Multilayered feed forward neural network based on particle swarm optimizer algorithm. J Syst Eng Electron 16(4):682-686.

[5]

Chang CC, Lin CJ (2001) LIBSVM: A Library for Support Vector Machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm

[6]

Chau KW (2007) Application of a PSO-based neural network in analysis of outcomes of construction claims. Autom Constr 16(5):642-646.

[7]

Civicioglu P, Besdok E (2011) A conceptual comparison of the Cuckoo-search, particle swarm optimization, differential evolution and artificial bee colony algorithms. Artif Intell Rev 39(5):315-346.

[8]

Da Y, Ge XR (2005) An improved PSO-based ANN with simulated annealing technique. Neurocomputing 63:527-533.

Digital Library

[9]

Deng W, Li W, Yang XH (2011) A novel hybrid optimization algorithm of computational intelligence techniques for highway passenger volume prediction. Expert Syst Appl 38(5):4198-4205.

Digital Library

[10]

Deng W, Chen R, Gao J, Song YJ, Xu JJ (2012) A novel parallel hybrid intelligence optimization algorithm for a function approximation problem. Comput Math Appl 63(1):325-336.

Digital Library

[11]

Fan SKS, Liang YC, Zahara E (2004) Hybrid simplex search and particle swarm optimization for the global optimization for multimodal functions. Eng Optim 36(5):401-418.

[12]

Goldberg DE (1989) Genetic algorithm in search optimization and machine learning. Addison-Wesley, New York.

[13]

Gori M, Tesi A (1992) On the problem of local minima in back-propagation. IEEE Trans Pattern Anal 14(1):76-86.

Digital Library

[14]

Holland J (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor.

[15]

Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(7):359-366.

Digital Library

[16]

Hsu CC, Gao CH (2008) Particle swarm optimization incorporating simplex search and center particle for global optimization. In: Proceedings of the IEEE conference on soft computing in industrial applications, pp 25-27.

[17]

Juang CF (2004) A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Trans Syst Man Cybern Part B Cybern 34(3):997-1006.

Digital Library

[18]

Juang CF, Chung IF, Hsu CH (2007) Automatic construction of feed-forward/recurrent fuzzy systems by clustering-aided simplex particle swarm optimization. Fuzzy Set Syst 158(18):1979-1996.

Digital Library

[19]

Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, pp 1942-1984.

[20]

Liu A, Zahaya E, Yang MT (2012) A modified NM-PSO method for parameter estimation problems of models. J Appl Math, p 2012.

[21]

Mendes R, Cortez P, Rocha M, Neves J (2002) Particle swarms for feed-forward neural network training. In: Proceedings of the international joint conference on neural networks, pp 1895-1899.

[22]

Maindonald J, John Braun W (2012) DAAG: Data Analysis And Graphics data and functions. http://CRAN.R-project.org/package=DAAG

[23]

Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7:308-313.

[24]

Nazareth L, Tseng P (2002) Gilding the lily: a variant of the Nelder-Mead algorithm based on golden-section search. Comput Optim Appl 22(1):133-144.

[25]

Rusiecki A (2007) Robust LTS backpropagation learning algorithm. Lect Notes Comput Sci 4507:102-109.

[26]

R Development Core Team (2011) R: A Language and Environment for Statistical Computing. http://www.R-project.org/

[27]

Shaw S, Kinsner W (1996) Chaotic simulated annealing in multilayer feedforward networks. In: Proceedings of Canadian conference on electrical and computer engineering, pp 265-269.

[28]

Salerno J (1997) Using the particle swarm optimization technique to train a recurrent neural model. In: Proceedings of ninth IEEE international conference tools with artificial intelligence, pp 45-49.

[29]

Trelea IC (2003) The particle swarm optimization algorithm: Convergence analysis and parameter selection. Inform Process Lett 85:317-325.

Digital Library

[30]

Tan Y, Ruan G (2010) A three-layer back-propagation neural network for spam detection using artificial immune concentration. Soft Comput 14(3):139-150.

[31]

Venkatesh YV, Kumar Raja S (2006) Multiple contour extraction from graylevel images using an artificial neural network. IEEE Trans Image Process 15(5):892-899.

Digital Library

[32]

Yu J, Xi L, Wang S (2007) An improved particle swarm optimization for evolving feedforward artificial neural networks. Neural Process Lett 26(4):217-231.

Digital Library

[33]

Zhang C, Shao H, Li Y (2000) Particle swarm optimization for evolving artificial neural network. In: Proceedings of the IEEE international conference on system, man, and cybernetics, pp 2487-2490.

[34]

Zhang JR, Zhang J, Lok TM, Lyu MR (2007) A hybrid particle swarm optimization back-propagation algorithm for feedforward neural network training. Appl Math Comput 185(3):1026-1037.

Digital Library

[35]

Zahaya E, Liu A (2010) Solving parameter identification problem by hybrid particle swarm optimization. In: Proceedings of the international multiconference of engineers and computer scientists, pp 36-38.

Cited By

Li SHuang XWang XLi Y(2022)A new reliability analysis approach with multiple correlation neural networks methodSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-022-07685-627:11(7449-7458)Online publication date: 12-Dec-2022
https://dl.acm.org/doi/10.1007/s00500-022-07685-6
Hamza FFerhat DAbderazek HDahane M(2022)A new efficient hybrid approach for reliability-based design optimization problemsEngineering with Computers10.1007/s00366-020-01187-538:3(1953-1976)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s00366-020-01187-5
Pang JHe JDong H(2019)Hybrid evolutionary programming using adaptive Lévy mutation and modified Nelder---Mead methodSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-018-3422-423:17(7913-7939)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1007/s00500-018-3422-4
Show More Cited By

Training neural networks via simplified hybrid algorithm mixing Nelder---Mead and particle swarm optimization methods

Recommendations

An improved cooperative particle swarm optimization and its application

A powerful cooperative evolutionary particle swarm optimization (PSO) algorithm based on two swarms with different behaviors to improve the global performance of PSO is proposed. In this method, one swarm tracks the best position and the other leaves ...
Accelerating particle swarm optimization using crisscross search

This paper introduces a novel crisscross search particle swarm optimizer called CSPSO.The CSPSO algorithm has significant superiority over most of the other PSO variants in terms of solution accuracy and convergence rate.The swarm in CSPSO is directly ...
Particle Swarm Optimization with Hybrid Velocity Updating Strategies
IITA '09: Proceedings of the 2009 Third International Symposium on Intelligent Information Technology Application - Volume 02

Particle Swarm Optimization (PSO) is a recently proposed population-based evolutionary algorithm, which shows good performance in many optimization problems. To achieve better performance, this paper presents a new variant of PSO algorithm called PSO ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Soft Computing - A Fusion of Foundations, Methodologies and Applications

Soft Computing - A Fusion of Foundations, Methodologies and Applications Volume 19, Issue 3

March 2015

287 pages

ISSN:1432-7643

EISSN:1433-7479

Issue’s Table of Contents

Copyright © Copyright © 2015 Springer-Verlag Berlin Heidelberg.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 March 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li SHuang XWang XLi Y(2022)A new reliability analysis approach with multiple correlation neural networks methodSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-022-07685-627:11(7449-7458)Online publication date: 12-Dec-2022
https://dl.acm.org/doi/10.1007/s00500-022-07685-6
Hamza FFerhat DAbderazek HDahane M(2022)A new efficient hybrid approach for reliability-based design optimization problemsEngineering with Computers10.1007/s00366-020-01187-538:3(1953-1976)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s00366-020-01187-5
Pang JHe JDong H(2019)Hybrid evolutionary programming using adaptive Lévy mutation and modified Nelder---Mead methodSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-018-3422-423:17(7913-7939)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1007/s00500-018-3422-4
Wang XShi YYan YGu X(2017)Intelligent welding robot path optimization based on discrete elite PSOSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2121-221:20(5869-5881)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s00500-016-2121-2
Holimchayachotikul PLeksakul K(2017)Predictive performance measurement system for retail industry using neuro-fuzzy system based on swarm intelligenceSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2082-521:7(1895-1912)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1007/s00500-016-2082-5
Sarakhsi MFatemi Ghomi SKarimi B(2016)A new hybrid algorithm of scatter search and Nelder-Mead algorithms to optimize joint economic lot sizing problemJournal of Computational and Applied Mathematics10.1016/j.cam.2015.07.027292:C(387-401)Online publication date: 15-Jan-2016
https://dl.acm.org/doi/10.1016/j.cam.2015.07.027

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents