Abstract
Motivation is a fundamental topic when implementing cognitive architectures aimed at lifelong open-ended learning in autonomous robots. In particular, it is of paramount importance for these types of architectures to be able to establish goals that provide purpose to the robot’s interaction with the world as well as to progressively learn value functions within its state space that allow reaching those goals whatever the starting point. This paper aims at exploring a developmental approach to the generation of high level neural network based value functions in complex continuous state spaces through a re-description process. This process starts by obtaining relatively simple Separable Utility Regions (SURs) which allow the system to consistently achieve goals, although not necessarily in the most efficient manner. The traces obtained by these SURs are then used to provide training data for a neural network based value function. Through a simple experiment with the Robobo robot, we show that this procedure can be more generalizable than attempting to directly obtain the value function through more traditional means.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sweller, J.: Evolution of human cognitive architecture. In: The Psychology of Learning and Motivation (2003)
Scott, P.D., Markovitch, S.: Learning novel domains through curiosity and conjecture. In: IJCAI, pp. 669–674 (1989)
Asada, M., et al.: Cognitive developmental robotics: a survey. IEEE Trans. Auton. Ment. Dev. 1(1), 12–34 (2009)
Karmiloff-Smith, A.: Beyond modularity: a developmental perspective on cognitive science. Behav. Brain Sci. (1992)
Carassa, A., Tirassa, M.: Representational redescription and cognitive architectures, pp. 711–712 (1994)
Maslow, A.H.: A theory of human motivation. Psychol. Rev. 50(13), 370–396 (1943)
Maslow, A.H.: Human motivation. Hum. Motiv. (1987)
Hull, C.L.: Principles of Behavioir. Appleton-Century-Crofts, New York (1943)
Kober, J., Bagnell, J.A., Peters, J.: Reinforcement learning in robotics: a survey. Int. J. Robot. Res. 32(11), 1238–1274 (2013)
Sutton, R.S., Barto, A.G.: Introduction to reinforcement learning. Learning 4(1996), 1–5 (1998)
Harlow, H.F.: Learning and satiation of response in intrinsically motivated complex puzzle performance by monkeys. J. Comp. Physiol. Psychol. 43(4), 289–294 (1950)
Ryan, R., Deci, E.: Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp. Educ. Psychol. 25(1), 54–67 (2000)
Kagan, J.: Motives and development. J. Pers. Soc. Psychol. 22(1), 51 (1972)
Oudeyer, P.-Y.: Intelligent adaptive curiosity: a source of self-development. Science 80 (2004)
Baldassarre, G., Mirolli, M.: Intrinsically motivated learning systems: an overview. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 1–14. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-32375-1_1
Friston, K.J., Tononi, G., Reeke, G.N., Sporns, O., Edelman, G.M.: Value-dependent selection in the brain: simulation in a synthetic neural model. Neuroscience 59, 229–243 (1994)
Rolf, M., Asada, M.: What are goals? And if so, how many? pp. 332–339 (2015)
Baldassarre, G., Stafford, T., Mirolli, M., Redgrave, P., Ryan, R., Barto, A.: Intrinsic motivations and open-ended development in animals, humans, and robots: an overview. Front. Psychol. 5, 1–8 (2014)
Huang, X., Weng, J.: Value system development for a robot. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 2883–2888 (2004)
Huang, X., Weng, J.: Novelty and reinforcement learning in the value system of developmental robots. In: Proceedings of the Second International Workshop on Epigenetic Robotics, pp. 47–55 (2002)
Zhang, Y., Weng, J.: Action chaining by a developmental robot with a value system. In: Proceedings of the 2nd International Conference on Development and Learning, ICDL 2002 (2002)
Salgado, R., Prieto, A., Bellas, F., Duro, R.J.: Motivational engine for cognitive robotics in non-static tasks. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Toledo Moreo, J., Adeli, H. (eds.) IWINAC 2017. LNCS, vol. 10337, pp. 32–42. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59740-9_4
Salgado, R., Prieto, A., Bellas, F., Calvo-Varela, L., Duro, R.J.: Motivational engine with autonomous sub-goal identification for the Multilevel Darwinist Brain. Biol. Inspir. Cogn. Archit. 17, 1–11 (2016)
Salgado, R., Prieto, A., Bellas, F., Calvo-Varela, L., Duro, R.J.: Neuroevolutionary motivational engine for autonomous robots. In: GECCO 2016 Companion - Proceedings of the 2016 Genetic and Evolutionary Computation Conference (2016)
EU DREAM H2020 Project. http://www.robotsthatdream.eu
Bellas, F., et al.: The Robobo project: bringing educational robotics closer to real-world applications. In: Lepuschitz, W., Merdan, M., Koppensteiner, G., Balogh, R., Obdržálek, D. (eds.) RiE 2017. AISC, vol. 630, pp. 226–237. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-62875-2_20
Acknowledgements
This work was partially funded by the EU’s H2020 research and innovation programme under grant agreement No. 640891 (DREAM project) and by the Xunta de Galicia and European Regional Development Funds under grant RedTEIC (ED341D R2016/012) and grant ED431C 2017/12.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Romero, A., Bellas, F., Prieto, A., Duro, R.J. (2018). A Re-description Based Developmental Approach to the Generation of Value Functions for Cognitive Robots. In: de Cos Juez, F., et al. Hybrid Artificial Intelligent Systems. HAIS 2018. Lecture Notes in Computer Science(), vol 10870. Springer, Cham. https://doi.org/10.1007/978-3-319-92639-1_56
Download citation
DOI: https://doi.org/10.1007/978-3-319-92639-1_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92638-4
Online ISBN: 978-3-319-92639-1
eBook Packages: Computer ScienceComputer Science (R0)