More Web Proxy on the site http://driver.im/

chapter

Intrinsically motivated machines

Authors:

Frédéric Kaplan,

Pierre-Yves OudeyerAuthors Info & Claims

50 years of artificial intelligence: essays dedicated to the 50th anniversary of artificial intelligence

January 2007

Pages 303 - 314

Published: 01 January 2007 Publication History

Abstract

Children seem intrinsically motivated to manipulate, to explore, to test, to learn and they look for activities and situations that provide such learning opportunities. Inspired by research in developmental psychology and neuroscience, some researchers have started to address the problem of designing intrinsic motivation systems. A robot controlled by such systems is able to autonomously explore its environment not to fulfil predefined tasks but driven by an incentive to search for situations where learning happens efficiently. In this paper, we present the origins of these intrinsically motivated machines, our own research in this novel field and we argue that intrinsic motivation might be a crucial step towards machines capable of life-long learning and open-ended development.

References

[1]

Barto, A., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: ICDL 2004. Proceedings of the 3rd International Conference on Development and Learning, Salk Institute, San Diego (2004)

[2]

Berlyne, D.: Conflict, Arousal and Curiosity. McGraw-Hill, New York (1960)

[3]

Cohn, D., Ghahramani, Z., Jordan, M.: Active learning with statistical models. Journal of artificial intelligence research 4, 129-145 (1996)

Digital Library

[4]

Csikszenthmihalyi, M.: Flow-the psychology of optimal experience. Harper Perennial (1991)

[5]

Dayan, P., Belleine, W.: Reward, motivation and reinforcement learning. Neuron 36, 285-298 (2002)

[6]

De Charms, R.: Personal causation: the internal affective determinants of behavior. Academic Press, New York (1968)

[7]

Deci, E., Ryan, R.: Intrinsic Motivation and Self-Determination in Human Behavior. Plenum Press, New York (1985)

[8]

Dember, W.N., Earl, R.W.: Analysis of exploratory, manipulatory and curiosity behaviors. Psychological Review 64, 91-96 (1957)

[9]

Drescher, G.L.: Made-up minds. MIT Press, Cambridge, MA (1991)

[10]

Fedorov, V.: Theory of Optimal Experiment. Academic Press, New York (1972)

[11]

Festinger, L.: A theory of cognitive dissonance. Evanston, Row, Peterson (1957)

[12]

Harlow, H.: Learning and satiation of response in intrinsically motivated complex puzzle performances by monkeys. Journal of Comparative and Physiological Psychology 43, 289-294 (1950)

[13]

Hebb, D.O.: Drives and the c.n.s (conceptual nervous system). Psychological review 62, 243-254 (1955)

[14]

Horvitz, J.-C.: Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96(4), 651-656 (2000)

[15]

Huang, X., Weng, J.: Novelty and reinforcement learning in the value system of developmental robots. In: Prince, C., Demiris, Y., Marom, Y., Kozima, H., Balkenius, C. (eds.) Proceedings of the 2nd international workshop on Epigenetic Robotics: Modeling cognitive development in robotic systems, Lund University Cognitive Studies 94, pp. 47-55 (2002)

[16]

Hull, C.L.: Principles of behavior: an introduction to behavior theory. Appleton-Century-Croft, New York (1943)

[17]

Hunt, J.M.: Intrinsic motivation and its role in psychological development. Nebraska symposium on motivation 13, 189-282 (1965)

[18]

Kagan, J.: Motives and development. Journal of Personality and Social Psychology 22, 51-66 (1972)

[19]

Kakade, S., Dayan, P.: Dopamine: Generalization and bonuses. Neural Networks 15, 549-559 (2002)

Digital Library

[20]

Kaplan, F., Oudeyer, P.-Y.: The progress-drive hypothesis: an interpretation of early imitation. In: Nehaniv, C., Dautenhahn, K. (eds.) Models and mechanisms of imitation and social learning: Behavioural, social and communication dimensions, pp. 361-377. Cambridge University Press, Cambridge (2007a)

[21]

Kaplan, F., Oudeyer, P.-Y.: Un robot motivé pour apprendre: le role des motivations intrinseques dans le development sensorimoteur. Enfance 59(1), 46-58 (2007b)

[22]

Marshall, J., Blank, D., Meeden, L.: An emergent framework for self-motivation in developmental robotics. In: ICDL 2004. Proceedings of the 3rd International Conference on Development and Learning, Salk Institute, San Diego (2004)

[23]

Minsky, M.: A framework for representing knowledge. In: Wiston, P. (ed.) The psychology of computer vision, pp. 211-277. Mc Graw Hill, New York (1975)

[24]

Montgomery, K.: The role of exploratory drive in learning. Journal of Comparative and Physiological Psychology 47, 60-64 (1954)

[25]

Oudeyer, P.-Y., Kaplan, F.: Discovering communication. Connection Science 18(2), 189-206 (2006)

[26]

Oudeyer, P.-Y., Kaplan, F., Hafner, V.: Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation 11(1), 265-286 (2007)

Digital Library

[27]

Panksepp, J.: Affective neuroscience: the foundations of human and animal emotions. Oxford University Press, Oxford (1998)

[28]

Piaget, J.: The origins of intelligence in children. Norton, New York (1952)

[29]

Schank, R., Abelson, R.: Scripts, plans, goals and understanding: An inquiry into human knowledge structures. Lawrence Erlbaum Associates, Hillsdale, NJ (1977)

[30]

Schmidhuber, J.: Curious model-building control systems. In: Proceeding International Joint Conference on Neural Networks, vol. 2, pp. 1458- 1463. IEEE, Singapore (1991)

[31]

Steels, L.: The autotelic principle. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS (LNAI), vol. 3139, pp. 231-242. Springer, Heidelberg (2004)

[32]

Sutton, R., Barto, A.: Reinforcement learning: an introduction. MIT Press, Cambridge, MA (1998)

Digital Library

[33]

Sutton, R., Precup, D., Singh, S.: Between mdpss and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112, 181-211 (1999)

Digital Library

[34]

Thrun, S., Pratt, L.: Learning to Learn. Kluwer Academic Publishers, Dordrecht (1998)

Digital Library

[35]

Turing, A.: Computing machinery and intelligence. Mind 59, 433-460 (1950)

Digital Library

[36]

White, R.: Motivation reconsidered: The concept of competence. Psychological review 66, 297-333 (1959)

Cited By

Helmes JTaylor ACao XHöök KSchmitt PVillar NGross MNunes NDo EBrewster SOakley I(2011)Rudiments 1, 2 & 3Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction10.1145/1935701.1935730(145-152)Online publication date: 22-Jan-2011
https://dl.acm.org/doi/10.1145/1935701.1935730
Lungarella MIida FBongard JPfeifer R(2007)AI in the 21st century - with historical reflections50 years of artificial intelligence10.5555/1806115.1806117(1-8)Online publication date: 1-Jan-2007
https://dl.acm.org/doi/10.5555/1806115.1806117

Intrinsically motivated machines
1. Computer systems organization
  1. Embedded and cyber-physical systems
2. Computing methodologies
  1. Artificial intelligence
    1. Control methods
    2. Planning and scheduling

Recommendations

Intrinsically motivated model learning for developing curious robots

Reinforcement Learning (RL) agents are typically deployed to learn a specific, concrete task based on a pre-defined reward function. However, in some cases an agent may be able to gain experience in the domain prior to being given a task. In such cases, ...
Intrinsic Motivation Systems for Autonomous Mental Development

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of ...
Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective

There is great interest in building intrinsic motivation into artificial systems using the reinforcement learning framework. Yet, what intrinsic motivation may mean computationally, and how it may differ from extrinsic motivation, remains a murky and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide books

50 years of artificial intelligence: essays dedicated to the 50th anniversary of artificial intelligence

January 2007

398 pages

ISBN:3540772952

Editors:
Max Lungarella
University of Zurich, Artificial Intelligence Laboratory, Zurich, Switzerland
,
Rolf Pfeifer
University of Zurich, Artificial Intelligence Laboratory, Zurich, Switzerland
,
Fumiya Iida
Massachusetts Institute of Technology, Robot Locomotion Group Computer Science and Artificial Intelligence Laboratory, Cambridge, MA
,
Josh Bongard
University of Vermont, Department of Computer Science, Burlington, VT

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 January 2007

Author Tags

Qualifiers

Chapter

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
5
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Helmes JTaylor ACao XHöök KSchmitt PVillar NGross MNunes NDo EBrewster SOakley I(2011)Rudiments 1, 2 & 3Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction10.1145/1935701.1935730(145-152)Online publication date: 22-Jan-2011
https://dl.acm.org/doi/10.1145/1935701.1935730
Lungarella MIida FBongard JPfeifer R(2007)AI in the 21st century - with historical reflections50 years of artificial intelligence10.5555/1806115.1806117(1-8)Online publication date: 1-Jan-2007
https://dl.acm.org/doi/10.5555/1806115.1806117

View Options

View options

Media

Figures

Other

Tables

View Table of Contents