More Web Proxy on the site http://driver.im/

research-article

Overcoming deception in evolution of cognitive behaviors

Authors:

Risto MiikkulainenAuthors Info & Claims

GECCO '14: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

Pages 185 - 192

https://doi.org/10.1145/2576768.2598300

Published: 12 July 2014 Publication History

Abstract

When scaling neuroevolution to complex behaviors, cognitive capabilities such as learning, communication, and memory become increasingly important. However, successfully evolving such cognitive abilities remains difficult. This paper argues that a main cause for such difficulty is deception, i.e. evolution converges to a behavior unrelated to the desired solution. More specifically, cognitive behaviors often require accumulating neural structure that provides no immediate fitness benefit, and evolution often thus converges to non-cognitive solutions. To investigate this hypothesis, a common evolutionary robotics T-Maze domain is adapted in three separate ways to require agents to communicate, remember, and learn. Indicative of deception, evolution driven by objective-based fitness often converges upon simple non-cognitive behaviors. In contrast, evolution driven to explore novel behaviors, i.e. novelty search, often evolves the desired cognitive behaviors. The conclusion is that open-ended methods of evolution may better recognize and reward the stepping stones that are necessary for cognitive behavior to emerge.

References

[1]

T. Aaltonen et al. Measurement of the top quark mass with dilepton events selected using neuroevolution at CDF. Physical Review Letters, 2009.

[2]

Todd S. Braver, Jonathan D. Cohen, and David Servan-Schreiber. A computational model of prefrontal cortex function. Advances in Neural Information Processing Systems: 7, 7: 141, 1995.

[3]

Thomas J. Carew, Edgar T. Walters, and Eric R. Kandel. Classical conditioning in a simple withdrawal reflex in aplysia californica. The Journal of Neuroscience, 1 (12): 1426--1437, 1981.

[4]

Stephen Jay Gould and Elisabeth S. Vrba. Exaptation-a missing term in the science of form. Paleobiology, pages 4--15, 1982.

[5]

Inman Harvey.biblink The Artificial Evolution of Adaptive Behavior http://www.cogs.susx.ac.uk/users/inmanh/inman_thesis.html. PhD thesis, School of Cognitive and Computing Sciences, University of Sussex, Sussex, 1993.

[6]

Gregory S. Hornby. ALPS: the age-layered population structure for reducing the problem of premature convergence. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2006), pages 815--822, New York, NY, USA, 2006. ACM.

Digital Library

[7]

Joel Lehman and Kenneth O. Stanley. Exploiting open-endedness to solve problems through the search for novelty. In Proc. of the Eleventh Intl. Conf. on Artificial Life (ALIFE XI), Cambridge, MA, 2008. MIT Press.

[8]

}lehman:ecj11Joel Lehman and Kenneth O. Stanley. Abandoning objectives: Evolution through the search for novelty alone. Evol. Comp., 19 (2): 189--223, 2011.

Digital Library

[9]

}lehman:gptpJoel Lehman and Kenneth O. Stanley. Novelty seach and the problem with objectives. In Genetic Programming in Theory and Practice IX (GPTP 2011), chapter 3, pages 37--56. Springer, 2011.

[10]

Joel Lehman, Kenneth O. Stanley, and Risto Miikkulainen. Effective diversity maintenance in deceptive domains. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2013). ACM, 2013.

Digital Library

[11]

Andrew P. Martin. Increasing genomic complexity by gene duplication and the origin of vertebrates. The American Naturalist, 154 (2): 111--128, 1999.

[12]

Ollion, Pinville, and Doncieux}ollion:emergenceCharles Ollion, Tony Pinville, and Stéphane Doncieux. Emergence of memory in neuroevolution: impact of selection pressures. In Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference companion, pages 369--372. ACM, 2012.

Digital Library

[13]

Ollion, Pinville, and Stéphane}ollion:memoryCharles Ollion, Tony Pinville, and Doncieux Stéphane. With a little help from selection pressures: evolution of memory in robot controllers. In Artificial Life, volume 13, pages 407--414, 2012.

[14]

David S. Olton. Mazes, maps, and memory. American Psychologist, 34 (7): 583, 1979.

[15]

Pierre-Yves Oudeyer. Intelligent adaptive curiosity: a source of self-development. In Proceedingsof the Fourth International Workshop on Epigenetic Robotics. Lund University Cognitive Studies, 2004.

[16]

Thomas S. Ray. Evolution, complexity, entropy and artificial reality. Physica D: Nonlinear Phenomena, 75 (1): 239--263, 1994.

Digital Library

[17]

Sebastian Risi, Charles E Hughes, and Kenneth O Stanley. Evolving plastic neural networks with novelty search. Adaptive Behavior, 18 (6): 470--491, 2010.

Digital Library

[18]

Sebastian Risi and Kenneth O Stanley. A unified approach to evolving plasticity and neural geometry. In Neural Networks (IJCNN), The 2012 International Joint Conference on, pages 1--8. IEEE, 2012.

[19]

Gregory M. Saunders and Jordan B. Pollack. The evolution of communication schemes over continuous channels. From Animals to Animats, 4: 580--589, 1996.

[20]

J. Schmidhuber. Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science, 18 (2): 173--187, 2006.

[21]

Andrea Soltoggio and Ben Jones. Novelty of behaviour as a basis for the neuro-evolution of operant reward learning. In Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pages 169--176. ACM, 2009.

Digital Library

[22]

Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen. Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation Special Issue on Evolutionary Computation and Games, 9 (6): 653--668, 2005.

Digital Library

[23]

Kenneth O. Stanley and Risto Miikkulainen.biblinkEvolving neural networks through augmenting topologies http://nn.cs.utexas.edu/keyword?stanley:ec02. Evolutionary Computation, 10: 99--127, 2002.

Digital Library

[24]

Kenneth O. Stanley and Risto Miikkulainen.biblinkCompetitive coevolution through evolutionary complexification http://nn.cs.utexas.edu/keyword?stanley:jair04. Journal of Artificial Intelligence Research, 21: 63--100, 2004.

Digital Library

[25]

Paul Tonelli and Jean-Baptiste Mouret. On the relationships between synaptic plasticity and generative systems. In Proceedings of the 13th annual conference on Genetic and evolutionary computation, pages 1531--1538. ACM, 2011.

Digital Library

[26]

Gregory M. Werner and Micheal G. Dyer. Evolution of communication in artificial organisms. In Proceedings of the Second International Conference of Artificial Life, pages 659--687, 1991.

[27]

Brian M. Yamauchi and Randall D. Beer. Sequential behavior and learning in evolved dynamical neural networks. Adaptive Behavior, 2 (3): 219--246, 1994.

Digital Library

[28]

Tom Ziemke and Mikael Thieme. Neuromodulation of reactive sensorimotor mappings as a short-term memory mechanism in delayed response tasks. Adaptive Behavior, 10 (3--4): 185--199, 2002.

Digital Library

Cited By

Shahrzad HHodjat BDollé CDenissov ALau SGoodhew DDyer JMiikkulainen R(2020)Enhanced Optimization with Composite Objectives and Novelty PulsationGenetic Programming Theory and Practice XVII10.1007/978-3-030-39958-0_14(275-293)Online publication date: 8-May-2020
https://doi.org/10.1007/978-3-030-39958-0_14
Nogueira Yde Brito CVidal CCavalcante-Neto J(2019)Towards intrinsic autonomy through evolutionary computationArtificial Intelligence Review10.1007/s10462-019-09798-1Online publication date: 17-Dec-2019
https://doi.org/10.1007/s10462-019-09798-1
Wei YHiraga MOhkura KCar Z(2019)Autonomous task allocation by artificial evolution for robotic swarms in complex tasksArtificial Life and Robotics10.1007/s10015-018-0466-624:1(127-134)Online publication date: 1-Mar-2019
https://dl.acm.org/doi/10.1007/s10015-018-0466-6
Show More Cited By

Index Terms

Overcoming deception in evolution of cognitive behaviors
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Accelerating the Evolution of Cognitive Behaviors Through Human-Computer Collaboration
GECCO '16: Proceedings of the Genetic and Evolutionary Computation Conference 2016

An open problem in neuroevolution (i.e. evolving artificial neural networks) is to evolve complex cognitive behaviors that allow robots to adapt and learn from past experience. While previous studies on the evolution of cognitive behaviors have shown ...
Open-ended evolution with multi-containers QD
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Evolution in nature has allowed biological systems to develop and survive in many different environments. It can be attributed to one of the major features of natural evolution: its open-endedness, that can be considered as the ability to continuously ...
Effective diversity maintenance in deceptive domains
GECCO '13: Proceedings of the 15th annual conference on Genetic and evolutionary computation

Diversity maintenance techniques in evolutionary computation are designed to mitigate the problem of deceptive local optima by encouraging exploration. However, as problems become more difficult, the heuristic of fitness may become increasingly ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '14: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

July 2014

1478 pages

ISBN:9781450326629

DOI:10.1145/2576768

Editor-in-chief:
Christian Igel
Ruhr University of Bochum, University of Copenhagen
,
General Chair:
Dirk V. Arnold
Dalhousie University

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '14

Sponsor:

SIGEVO

GECCO '14: Genetic and Evolutionary Computation Conference

July 12 - 16, 2014

BC, Vancouver, Canada

Acceptance Rates

GECCO '14 Paper Acceptance Rate 180 of 544 submissions, 33%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)6

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shahrzad HHodjat BDollé CDenissov ALau SGoodhew DDyer JMiikkulainen R(2020)Enhanced Optimization with Composite Objectives and Novelty PulsationGenetic Programming Theory and Practice XVII10.1007/978-3-030-39958-0_14(275-293)Online publication date: 8-May-2020
https://doi.org/10.1007/978-3-030-39958-0_14
Nogueira Yde Brito CVidal CCavalcante-Neto J(2019)Towards intrinsic autonomy through evolutionary computationArtificial Intelligence Review10.1007/s10462-019-09798-1Online publication date: 17-Dec-2019
https://doi.org/10.1007/s10462-019-09798-1
Wei YHiraga MOhkura KCar Z(2019)Autonomous task allocation by artificial evolution for robotic swarms in complex tasksArtificial Life and Robotics10.1007/s10015-018-0466-624:1(127-134)Online publication date: 1-Mar-2019
https://dl.acm.org/doi/10.1007/s10015-018-0466-6
Huang PSentis LLehman JFok CMok AMiikkulainen R(2018)Tradeoffs in Neuroevolutionary Learning-Based Real-Time Robotic Task Design in the Imprecise Computation FrameworkACM Transactions on Cyber-Physical Systems10.1145/31789033:2(1-29)Online publication date: 10-Oct-2018
https://dl.acm.org/doi/10.1145/3178903
Soltoggio AStanley KRisi S(2018)Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networksNeural Networks10.1016/j.neunet.2018.07.013108(48-67)Online publication date: Dec-2018
https://doi.org/10.1016/j.neunet.2018.07.013
Risi STogelius J(2017)Neuroevolution in Games: State of the Art and Open ChallengesIEEE Transactions on Computational Intelligence and AI in Games10.1109/TCIAIG.2015.24945969:1(25-41)Online publication date: Mar-2017
https://doi.org/10.1109/TCIAIG.2015.2494596
Silva FDuarte MCorreia LOliveira SChristensen A(2016)Open issues in evolutionary roboticsEvolutionary Computation10.1162/EVCO_a_0017224:2(205-236)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1162/EVCO_a_00172
Miikkulainen RNeumann FSutton A(2016)Evolving Neural NetworksProceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion10.1145/2908961.2926977(229-253)Online publication date: 20-Jul-2016
https://dl.acm.org/doi/10.1145/2908961.2926977
Rawal AMiikkulainen RNeumann FSutton A(2016)Evolving Deep LSTM-based Memory Networks using an Information Maximization ObjectiveProceedings of the Genetic and Evolutionary Computation Conference 201610.1145/2908812.2908941(501-508)Online publication date: 20-Jul-2016
https://dl.acm.org/doi/10.1145/2908812.2908941
Greve RJacobsen ERisi SNeumann FSutton A(2016)Evolving Neural Turing Machines for Reward-based LearningProceedings of the Genetic and Evolutionary Computation Conference 201610.1145/2908812.2908930(117-124)Online publication date: 20-Jul-2016
https://dl.acm.org/doi/10.1145/2908812.2908930
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents