Abstract
Research into generative representations of complex data is a rapidly expanding field in machine learning. In this work we propose and evaluate a process of production and analysis of informative low-dimensional latent representations of real-world images with neural network models of unsupervised generative learning. A model of convolutional autoencoder based on VGG-16 architecture was used to produce low-dimensional generative representations of two datasets of aerial images and the characteristics of distributions of several classes of images of terrain studied. An analysis of latent distributions of terrain classes demonstrated a landscape of compact density structures for most studied classes with good separation of concept regions. The results of this work can be used in developing methods of effective learning in problems and environments with a deficit of labeled data based on the concept-sensitive structure in the latent representations that emerges in the process of unsupervised generative self-learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward neural networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)
Coates, A., Lee, H., Ng, A.Y.: An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of 14th International Conference on Artificial Intelligence and Statistics, Ft. Lauderdale, USA, vol. 15, pp. 215–223 (2011)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Welling, M., Kingma, D.P.: An introduction to variational autoencoders. Found. Trends Mach. Learn. 12(4), 307–392 (2019)
Le, Q.V.: A tutorial on deep learning: autoencoders, convolutional neural networks and recurrent neural networks. Stanford University (2015)
Kavukcuoglu, K., Sermanet, P., Boureau, Y.L., Gregor, K., Mathieu, M., Cun, Y.: Learning convolutional feature hierarchies for visual recognition. In: 23rd International Conference on Neural Information Processing Systems, Vancouver, Canada, vol. 1, pp. 1090–1098 (2010)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556 (2014)
Gondara, L.: Medical image denoising using convolutional denoising autoencoders. In: 16th IEEE International Conference on Data Mining Workshops (ICDMW), Barcelona, Spain, pp. 241–246 (2016)
Chandar, A.P.S., et al.: An autoencoder approach to learning bilingual word representations. In: 27th International Conference on Neural Information Processing Systems, Montreal, Canada, vol. 2, pp. 1853–1861 (2014)
Seddigh, N., Nandy, B., Bennett, D., Ren, Y., Dolgikh, S., et al.: A framework and system for classification of encrypted network traffic using machine learning. In: 2019 15th International Conference on Network and Service Management (CNSM-2019), pp. 1–5 (2019)
Le, Q.V., Ransato, M.A., Monga, R., et al.: Building high-level features using large scale unsupervised learning. https://arxiv.org/abs/1112.6209 (2012)
Banino, A., Barry, C., Kumaran, D.: Vector-based navigation using grid-like representations in artificial agents. Nature 557, 429–433 (2018)
Dolgikh, S.: Low-dimensional representations in unsupervised generative models. In: 20th International Conference Information Technologies - Applications and Theory (ITAT 2020), Slovakia, vol. 2718, pp. 239–245. CEUR-WS.org (2020)
Higgins, I., Matthey, L., Glorot, X., Pal, A., et al.: Early visual concept learning with unsupervised deep learning. https://arxiv.org/abs/1606.05579 (2016)
Marfil, R., Molina-Tanco, L., Bandera, A., Rodriguez, J.A., Sandoval, F.: Pyramid segmentation algorithms revisited. Pattern Recogn. 39(8), 1430–1451 (2006)
Chyrkov, A., Prystavka, P.: Suspicious object search in airborne camera video stream. In: Hu, Z., Petoukhov, S., Dychka, I., He, M. (eds.) Advances in Computer Science for Engineering and Education, vol. 754, pp. 340–348. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91008-6_34
Prystavka, P., Cholyshkina, O., Dolgikh, S., Karpenko, D.: Automated object recognition system based on convolutional autoencoder. In: 10th International Conference on Advanced Computer Information Technologies (ACIT-2020), Deggendorf, Germany, pp. 830–833 (2020)
Keras: The Python Deep Learning library. https://keras.io
Ester, M., Kriegel, H.-P., Sander, J., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: 2nd International Conference on Knowledge Discovery and Data Mining (KDD-96) Portland, USA, pp. 226–231 (1996)
Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory 21(1), 32–40 (1975)
Jolliffe, I.T.: Principal Component Analysis, 2nd edn., p. 487, 28. Springer, Cham (2002). https://doi.org/10.1007/b98835
Dolgikh, S.: Categorized representations and general learning. In: Aliev, R., Kacprzyk, J., Pedrycz, W., Jamshidi, M., Babanli, M., Sadikoglu, F. (eds.) 10th International Conference on Theory and Application of Soft Computing, Computing with Words and Perceptions, Prague, Czech Republic, vol. 1095, pp. 93–100. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35249-3_11
Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46(3), 175–185 (1992)
Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length and Helmholtz free energy. Adv. Neural Inf. Process. Syst. 6, 3–10 (1994)
Ranzato, M.A., Boureau, Y.-L., Chopra, S., LeCun, Y.: A unified energy-based framework for unsupervised learning. In: 11th International Conference on Artificial Intelligence and Statistics (AISTATS), San Huan, Puerto Rico, vol. 2, pp. 371–379 (2007)
Zhou, X., Belkin, M.: Semi-supervised learning. In: Academic Press Library in Signal Processing, vol. 1, pp. 1239–1269. Elsevier (2014)
Yoshida, T., Ohki, K.: Natural images are reliably represented by sparse and variable populations of neurons in visual cortex. Nat. Commun. 11, 872 (2020)
Bao, X., Gjorgiea, E., Shanahan, L.K., Howard, J.D., Kahnt, T., Gottfried, J.A.: Grid-like neural representations support olfactory navigation of a two-dimensional odor space. Neuron 102(5), 1066–1075 (2019)
Hassabis, D., Kumaran, D., Summerfield, C., Botvinick, M.: Neuroscience inspired artificial intelligence. Neuron 95, 245–258 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Prystavka, P., Dolgikh, S., Cholyshkina, O., Kozachuk, O. (2022). Learned and Native Concepts in Latent Representations of Terrain Images. In: Ermolayev, V., et al. Information and Communication Technologies in Education, Research, and Industrial Applications. ICTERI 2021. Communications in Computer and Information Science, vol 1698. Springer, Cham. https://doi.org/10.1007/978-3-031-20834-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-20834-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20833-1
Online ISBN: 978-3-031-20834-8
eBook Packages: Computer ScienceComputer Science (R0)