Abstract
This paper presents two methods of using the dynamic momentum and learning rate adaption, to improve learning performance in spiking neural networks where neurons are modelled as spiking multiple times. The optimum value for the momentum factor is obtained from the mean square error with respect to the gradient of synaptic weights in the proposed algorithm. The delta-bar-delta rule is employed as the learning rate adaptation method. The XOR and Wisconsin breast cancer (WBC) classification tasks are used to validate the proposed algorithms. Results demonstrate no error and a minimal error of 0.08 are achieved for the XOR and WBC classification tasks respectively, which are better than the original Booij’s algorithm. The minimum number of epochs for XOR and Wisconsin breast cancer tasks are 35 and 26 respectively, which are also faster than the original Booij’s algorithm – i.e. 135 (for XOR) and 97 (for WBC). Compared with the original algorithm with static momentum and learning rate, the proposed dynamic algorithms can control the convergence rate and learning performance more effectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)
Liu, J., Harkin, J., Mcdaid, L., Halliday, D.M., Tyrrell, A.M., Timmis, J.: Self-repairing mobile robotic car using astrocyte-neuron networks. In: International Joint Conference on Neural Networks, pp. 1–8 (2016)
Bohte, S.M., Kok, J.N., La Poutré, H.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1–4), 17–37 (2002)
Xin, J., Embrechts, M.J.: Supervised learning with spiking neural networks. In: International Joint Conference on Neural Networks, vol. 3, no. 3, pp. 1772–1777 (2001)
McKennoch, S., Liu, D.L.D., Bushnell, L.G.: Fast modifications of the spikeprop algorithm. In: Proceedings of the 2006 IEEE International Joint Conference on Neural Networks, vol. 16, no. 6, pp. 3970–3977 (2006)
Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Netw. 1(4), 295–307 (1988)
Schrauwen, B.: Extending spikeprop. In: International Joint Conference on Neural Networks, vol. 1, no. 7, pp. 471–476 (2004)
Booij, O., Tat Nguyen, H.: A gradient descent rule for spiking neurons emitting multiple spikes. Inf. Process. Lett. 95(6), 552–558 (2005)
Kulkarni, S., Simon, S.P., Sundareswaran, K.: A spiking neural network (SNN) forecast engine for short-term electrical load forecasting. Appl. Soft Comput. J. 13(8), 3628–3635 (2013)
Rosado-Muñoz, A., Bataller-Mompeán, M., Guerrero-Martínez, J.: FPGA implementation of spiking neural networks. In: Proceedings of the 1st IFAC Conference on Embedded Systems, Computational Intelligence and Telematics in Control, vol. 45, no. 4, pp. 139–144 (2012)
Rosado-Muñoz, A., Bataller-Mompeán, M., Guerrero-Martínez, J.: FPGA implementation of spiking neural networks supported by a software design environment. IFAC Proc. Vol. 45(4), 1934–1939 (2012)
Awadalla, M.H.A., Sadek, M.A.: Spiking neural network-based control chart pattern recognition. Alex. Eng. J. 51(1), 27–35 (2012)
Dorogyy, Y., Kolisnichenko, V.: Designing spiking neural networks. In: Modern Problems of Radio Engineering, Telecommunications and Computer Science, vol. 6, pp. 124–127 (2016)
Ghosh-Dastidar, S., Adeli, H.: A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Netw. 22(10), 1419–1431 (2009)
Ghosh-Dastidar, S., Adeli, H.: Improved spiking neural networks for EEG classification and epilepsy and seizure detection. Integr. Comput. Aided Eng. 14(4), 187–212 (2007)
Kim, E.-M., Park, S.-M., Kim, K.-H., Lee, B.-H.: An effective machine learning algorithm using momentum scheduling. In: Fourth International Conference on Hybrid Intelligent Systems (HIS 2004), pp. 442–443 (2004)
Delshad, E., Moallem, P., Monadjemi, S.H.: Spiking neural network learning algorithms: using learning rates adaptation of gradient and momentum steps. In: 2010 5th International Symposium on Telecommunications, no. 1, pp. 944–949 (2010)
Chandra, B., Sharma, R.K.: Deep learning with adaptive learning rate using laplacian score. Expert Syst. Appl. 63(5), 1–7 (2016)
Huijuan, F., Jiliang, L., Fei, W.: Fast learning in spiking neural networks by learning rate adaptation. Chin. J. Chem. Eng. 20(6), 1219–1224 (2012)
Salomon, R., Van Hemmen, J.L.: Accelerating backpropagation through dynamic self-adaptation. Neural Netw. 9(4), 589–601 (1996)
Wolberg, W.H., Mangasarian, O.L.: Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proc. Nat. Acad. Sci. 87(12), 9193–9196 (1990)
Acknowledgement
This research was supported by the National Natural Science Foundation of China under Grant 61603104 and 61661008, the Guangxi Natural Science Foundation under Grant 2015GXNSFBA139256, 2016GXNSFCA380017 and 2014GXNSFBA118271, the funding of Overseas 100 Talents Program of Guangxi Higher Education, the Research Project of Guangxi University of China under Grant KY2016YB059, Guangxi Key Lab of Multi-source Information Mining & Security under Grant MIMS15-07 and MIMS14-04, and the Doctoral Research Foundation of Guangxi Normal University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Luo, Y., Fu, Q., Liu, J., Harkin, J., McDaid, L., Cao, Y. (2017). An Extended Algorithm Using Adaptation of Momentum and Learning Rate for Spiking Neurons Emitting Multiple Spikes. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2017. Lecture Notes in Computer Science(), vol 10305. Springer, Cham. https://doi.org/10.1007/978-3-319-59153-7_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-59153-7_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59152-0
Online ISBN: 978-3-319-59153-7
eBook Packages: Computer ScienceComputer Science (R0)