Abstract
In this paper, we propose a new type of information-theoretic method for competitive learning based, upon mutual information between competitive units and input patterns. In addition, we extend this method to a case where cooperation between competitive units exists to realize self-organizing maps. In computational methods, free energy is introduced to simplify the computation of mutual information. We applied our method to two problems, namely, the Senate data and ionosphere data problems. In both, experimental results confirmed that better performance could be obtained in terms of quantization and topographic errors. We also found that the information-theoretic methods tended to produce more equi-probable distribution of competitive units.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kamimura, R.: Information-theoretic competitive learning with inverse Euclidean distance output units. Neural Processing Letters 18, 163–184 (2003)
Torkkola, K.: Feature extraction by non-parametric mutual information maximization. Journal of Machine Learning Research 3, 1415–1438 (2003)
Nenadic, Z.: Information discriminant analysis: Feature extraction with an information-theoretic objective. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(8), 1394–1407 (2007)
Rose, K., Gurewitz, E., Fox, G.C.: Statistical mechanics and phase transition in clustering. Physical review letters 65(8), 945–948 (1990)
Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (1995)
Kamimura, R., Kamimura, T., Shultz, T.R.: Information theoretic competitive learning and linguistic rule acquisition. Transactions of the Japanese Society for Artificial Intelligence 16(2), 287–298 (2001)
Kamimura, R., Kamimura, T., Uchida, O.: Flexible feature discovery and structural information control. Connection Science 13(4), 323–347 (2001)
Romesburg, H.C.: Cluster Analysis for Researchers. Krieger Publishing Company, Florida (1984)
Vesanto, J., Himberg, J., Alhoniemi, E., Parhankangas, J.: SOM toolbox for Matlab. tech. rep., Laboratory of Computer and Information Science, Helsinki University of Technology (2000)
Kiviluoto, K.: Topology preservation in self-organizing maps. In: Proceedings of the IEEE International Conference on Neural Networks, pp. 294–299 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kamimura, R. (2010). Information-Theoretic Competitive and Cooperative Learning for Self-Organizing Maps. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds) Neural Information Processing. Models and Applications. ICONIP 2010. Lecture Notes in Computer Science, vol 6444. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17534-3_52
Download citation
DOI: https://doi.org/10.1007/978-3-642-17534-3_52
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17533-6
Online ISBN: 978-3-642-17534-3
eBook Packages: Computer ScienceComputer Science (R0)