Abstract
In hierarchical learning machines such as neural networks, Bayesian learning provides better generalization performance than maximum likelihood estimation. However, its accurate approximation using Markov chain Monte Carlo (MCMC) method requires huge computational cost. The exchange Monte Carlo (EMC) method was proposed as an improved algorithm of MCMC method. Although its effectiveness has been shown not only in Bayesian learning but also in many fields, the mathematical foundation of EMC method has not yet been established. In this paper, we clarify the asymptotic behavior of symmetrized Kullback divergence and average exchange ratio, which are used as criteria for designing the EMC method.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Atiyah, M.F.: Resolution of singularities and division of distributions. Communications of Pure and Applied Mathematics 13, 145–150 (1970)
Berg, B.A., Neuhaus, T.: Multicanonical algorithms for first order phase transitions. Physics Letter B 267(2), 249–253 (1991)
Hukushima, K.: Domain Wall Free Energy of Spin Glass Models: Numerical Method and Boundary Conditions. Physical Review E 60, 3606–3613 (1999)
Hukushima, K., Nemoto, K.: Exchange Monte Carlo Method and Application to Spin Glass Simulations. Journal of Physical Society of Japan 65(6), 1604–1608 (1996)
Hukushima, K.: Extended ensemble Monte Carlo approach to hardly relaxing problems. Computer Physics Communications 147, 77–82 (2002)
Iba, Y.: Extended Ensemble Monte Carlo. International Journal of Modern Physics C 12, 623–656 (2001)
Liang, F.: An effective Bayesian neural network classifier with a comparison study to support vector machine. Neural Computation 15, 1959–1989 (2003)
Marinari, E., Parisi, G.: Simulated tempering: a new Monte Carlo scheme. Europhysics Letters 19(6), 451–455 (1992)
Nagata, K., Watanabe, S.: Analysis of Exchange Ratio for Exchange Monte Carlo Method. In: FOCI’07. Proc. of The First IEEE Symposium on Foundation of Computational Intelligence, pp. 434–439. IEEE Computer Society Press, Los Alamitos (2007)
Pinn, K., Wieczerkowski, C.: Number of Magic Squares from Parallel Tempering Monte Carlo. International Journal of modern Physics 9(4), 541–546 (1998)
Sugita, Y., Okamoto, Y.: Replica-exchange molecular dynamics method for protein folding. Chemical Physics Letters 314(1-2), 141–151 (1999)
Watanabe, S.: Algebraic Analysis for Nonidentifiable Learning Machines. Neural Computation 13, 899–933 (2001)
Yamazaki, K., Watanabe, S.: Singularities in mixture models and upper bounds of stochastic complexity. Neural networks 16(7), 1029–1038 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nagata, K., Watanabe, S. (2007). Algebraic Geometric Study of Exchange Monte Carlo Method. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_70
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_70
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)