Abstract
In this paper, we examine a geometrical projection algorithm for statistical inference. The algorithm is based on Pythagorean relation and it is derivative-free as well as representation-free that is useful in nonparametric cases. We derive a bound of learning rate to guarantee local convergence. In special cases of m-mixture and e-mixture estimation problems, we calculate specific forms of the bound that can be used easily in practice.
Supported by JSPS KAKENHI Grant Number 17H01793, 19K12111.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Details of the proof will appear in the full version of the paper [3].
References
Akaho, S.: The e-PCA and m-PCA: Dimension reduction of parameters by information geometry. In: Proceedings of the 2004 IEEE International Joint Conference on Neural Networks, 2004, vol. 1, pp. 129–134. IEEE (2004)
Akaho, S.: Dimension reduction for mixtures of exponential families. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008. LNCS, vol. 5163, pp. 1–10. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87536-9_1
Akaho, S., Hino, H., Murata, N.: On a convergence property of a geometrical algorithm for statistical manifolds. arXiv preprint arXiv:1909.12644 [cs.LG] (2019)
Akaho, S., Hino, H., Nara, N., Murata, N.: Geometrical formulation of the nonnegative matrix factorization. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11303, pp. 525–534. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04182-3_46
Amari, S.: Information Geometry and Its Applications, vol. 194. Springer, Tokyo (2016). https://doi.org/10.1007/978-4-431-55978-8
Ay, N., Jost, J., Vân Lê, H., Schwachhöfer, L.: Information geometry, vol. 64. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56478-4
Fujiwara, A., Amari, S.: Gradient systems in view of information geometry. Phys. D Nonlinear Phenom. 80(3), 317–327 (1995)
Hino, H., Akaho, S., Murata, N.: An entropy estimator based on polynomial regression with Poisson error structure. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016, Part II. LNCS, vol. 9948, pp. 11–19. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46672-9_2
Hino, H., Koshijima, K., Murata, N.: Non-parametric entropy estimators based on simple linear regression. Comput. Stat. Data Anal. 89, 72–84 (2015). https://doi.org/10.1016/j.csda.2015.03.011. http://www.sciencedirect.com/science/article/pii/S0167947315000791
Lebanon, G., et al.: Riemannian Geometry and Statistical Machine Learning. LAP LAMBERT Academic Publishing (2015)
Murata, N., Fujimoto, Y.: Bregman divergence and density integration. J. Math-for-Ind. (JMI) 1(B), 97–104 (2009)
Nagaoka, H., Amari, S.: Differential geometry of smooth families of probability distributions. Technical report METR 82–7, University of Tokyo (1982)
Pistone, G.: Nonparametric information geometry. In: Nielsen, F., Barbaresco, F. (eds.) GSI 2013. LNCS, vol. 8085, pp. 5–36. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40020-9_3
Popoviciu, T.: Sur les équations algébriques ayant toutes leurs racines réelles. Mathematica (Cluj) 9, 129–145 (1935)
Takano, K., Hino, H., Akaho, S., Murata, N.: Nonparametric e-mixture estimation. Neural Comput. 28(12), 2687–2725 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Akaho, S., Hino, H., Murata, N. (2019). On a Convergence Property of a Geometrical Algorithm for Statistical Manifolds. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Communications in Computer and Information Science, vol 1143. Springer, Cham. https://doi.org/10.1007/978-3-030-36802-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-36802-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36801-2
Online ISBN: 978-3-030-36802-9
eBook Packages: Computer ScienceComputer Science (R0)