Abstract
Recently models of neural networks that can directly deal with complex numbers, complex-valued neural networks, have been proposed and several studies on their abilities of information processing have been done. One of the important factors to characterize behavior of a complex-valued neural network is its activation function which is a nonlinear complex function. This paper discusses the properties of activation functions from the standpoint of existence of an energy function for complex-valued neural networks. Two classes of complex functions which are widely used as activation functions in the models of complex-valued neural networks are considered. We investigate the properties of activation functions which assure existence of energy functions and discuss about how to find out complex functions which satisfy the properties.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
J. J. Hopfield, “Neurons with graded response have collective computational properties like those of two-state neurons”, Proc. Natl. Acad. of Sci. U.S.A., Vol.81, pp. 3088–3092, 1984.
J. J. Hopfield and D. W. Tank, ““Neural” Computation of Decisions in Optimization Problems”, Biol. Cybern., Vol. 81, pp. 141–152, 1985.
Y. Kuroe, N. Hashimoto and T. Mori, “On Energy Function for Complex-valued Neural Networks and Its Application”, CD-ROM Proc. of 9th International Conference on Neural Information Processing (ICONIP’02) Nov., 2002.
H. Leung and S. Haykin, “The Complex Backpropagation Algorithm”, IEEE Transactions on Signal Processing, Vol.39, pp. 2101–2104, 1991.
D. L. Birx, S. J. Pipenberg, “Chaotic Oscillators and Complex Mapping Feed Forward Networks (CMFFNS) for Signal Detection in Noisy Environments”, Proc. IEEE IJCNN’92, Vol. II, pp. 881–888, 1992.
A.J. Noest, “Phaser Neural Network,” Neural Information Processing Systems, pp. 584–591, D.Z. Anderson,ed., AIP, New York, 1988.
A. J. Noest, “Associative Memory in Sparse Phasor Neural Networks”, Europhysics Letters, Vol. 6,No. 6, pp. 469–474, 1988.
G. M. Georgiou and C. Koutsougeras, “Complex Domain Backpropagation”, IEEE Transactions on Circuits and Systems-II, Vol. 39,No. 5, pp. 330–334, 1992.
N. Benvenuto and F. Piazza, “On the Complex Backpropagation Algorithm” IEEE Transactions on Signal Processing, Vol.40, pp. 967–969, 1992.
G. Kechriotis and E. S. Manolakos, “Training Fully Recurrent Neural Networks with Complex Weights”, IEEE Transactions on Circuits and Systems-II, Vol.41,No.3, pp. 235–238, 1994.
M. Kinouchi and M. Hagiwara, “Learning Temporal Sequences by Complex Neurons with Local Feedback”, Proc. IEEE ICNN’ 95, Vol. VI, pp. 3165–3169, 1995.
A. Hirose, “Applications of Complex-Valued Neural Networks to Coherent Optical Computing Using Phase-Sensitive Detection Scheme”, Information Sciences-Applications, Vol. 2,No. 2, pp. 103–117, 1994.
N. Hashimoto, Y. Kuroe and T. Mori, “Theoretical Study of Qualitative Behaviors of Self-Correlation Type Associative Memory on Complex-valued Neural Networks,” Trans. of IEICE, Vol.J83-A,No.6, pp.750–760, June 2000 (in Japanese).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kuroe, Y., Yoshid, M., Mori, T. (2003). On Activation Functions for Complex-Valued Neural Networks — Existence of Energy Functions —. In: Kaynak, O., Alpaydin, E., Oja, E., Xu, L. (eds) Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003. ICANN ICONIP 2003 2003. Lecture Notes in Computer Science, vol 2714. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44989-2_117
Download citation
DOI: https://doi.org/10.1007/3-540-44989-2_117
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40408-8
Online ISBN: 978-3-540-44989-8
eBook Packages: Springer Book Archive